latent.stats.trajectory¶
Trajectory distance metrics for tool-call and conversation sequences (FR-5.3).
Functions¶
conversational_edit_distance¶
conversational_edit_distance(actual_turns: list[Turn], reference_turns: list[Turn], embedding_fn: Callable | None = None) -> MetricResult
Edit distance between conversation turn sequences with semantic cost.
The point_estimate is normalised:
1 - (edit_distance / max(len(actual), len(reference))).
Substitution costs: - Role mismatch: infinity (treated as insert + delete) - Same role, no embedding_fn: 1.0 if content differs, 0.0 if identical - Same role, with embedding_fn: 1 - cosine_similarity
Args: actual_turns: The actual conversation turns. reference_turns: The reference/ideal turns. embedding_fn: Optional function mapping content string to a vector.
Returns: MetricResult with method="conv_edit_distance".
tool_trajectory_distance¶
Levenshtein distance between tool-call sequences, normalised to [0, 1].
The returned point_estimate is 1 - (edit_distance / max(len(actual), len(ideal))),
so 1.0 means identical sequences and 0.0 means completely different.
A warning is emitted when the actual trajectory is more than twice the length of the ideal, suggesting possible loops.
Args: actual: Ordered list of tool names actually called. ideal: Ordered list of expected tool names.
Returns: MetricResult with method="levenshtein".