latent.flows.comparison_flow

comparison_flow — Score and compare against a baseline (FR-3.2).

Functions

comparison_flow

comparison_flow(eval_data: pd.DataFrame, judge: Any, baseline_scores: dict[str, np.ndarray] | None = None, gates: dict[str, float | GateSpec] | None = None, score_types: dict[str, str] | None = None, rubrics: dict | None = None, n_resamples: int = 10000, confidence_level: float = 0.95, seed: int | None = None) -> dict[str, Any]

Score a dataset and compare against baseline scores.

Args: eval_data: DataFrame with columns matching judge's prompt_template. judge: A Judge[T] instance. baseline_scores: Optional dict of baseline score arrays per metric. If None, comparison is skipped (first-run mode). gates: Optional quality gates. score_types: Optional override for score type detection. rubrics: Optional override for rubric detection. n_resamples: Bootstrap resamples. confidence_level: CI confidence level. seed: Random seed.

Returns: Dict with: scored_data, report, markdown, all_passed.