latent.flows.text_to_sql_flow

text_to_sql_flow — Evaluate text-to-SQL generation quality.

Functions

text_to_sql_flow

text_to_sql_flow(eval_data: pd.DataFrame, dialect: str = 'ansi', connection: Any | None = None, numeric_tolerance: float = 0.0, gold_results_column: str | None = None, predicted_results_column: str | None = None, gates: dict[str, float] | None = None, confidence_level: float = 0.95, n_resamples: int = 10000, seed: int | None = None) -> dict[str, Any]

Evaluate text-to-SQL generation with validity and result matching.

The input DataFrame must have a 'generated_sql' column. Optionally, provide gold_results_column and predicted_results_column for result set matching.

Args: eval_data: DataFrame with 'generated_sql' column. Optionally 'gold_sql' for reference queries. dialect: SQL dialect for parsing validation. connection: Optional DB-API 2.0 connection for execution checks. numeric_tolerance: Tolerance for approximate numeric matching. gold_results_column: Column containing expected result sets (list[dict]). predicted_results_column: Column containing actual result sets (list[dict]). gates: Quality thresholds for metrics. confidence_level: CI confidence level. n_resamples: Bootstrap resamples. seed: Random seed.

Returns: Dict with: metrics, all_passed, report.