latent.scores.sql

SQL evaluation scoring functions.

Functions

result_set_match_extended

result_set_match_extended(actual_sets: list[list[dict]], expected_sets: list[list[dict]], numeric_tolerance: float = 0.0, confidence_level: float = 0.95, n_resamples: int = 10000, seed: int | None = None) -> dict[str, MetricResult]

Extended result set matching with fuzzy numerics and row-level F1.

Args: actual_sets: List of actual result sets (each a list of row dicts). expected_sets: List of expected result sets (same length). numeric_tolerance: Absolute tolerance for numeric comparisons. 0.0 means exact match (default). confidence_level: CI confidence level. n_resamples: Bootstrap resamples. seed: Random seed.

Returns: Dict with metrics: - "exact_match": Binary exact set match (using existing result_set_match logic) - "row_f1": Per-query F1 score (how many expected rows found in actual)

sql_validity

sql_validity(queries: list[str], dialect: str = 'ansi', connection: Any | None = None, confidence_level: float = 0.95) -> dict[str, MetricResult]

Check SQL validity: parse check + optional execution check.

Args: queries: List of SQL query strings to validate. dialect: SQL dialect for parsing (default "ansi"). Passed to sqlglot. connection: Optional DB-API 2.0 connection for execution validation. Each query is executed inside a savepoint that is rolled back, so no data is mutated. confidence_level: Confidence level for Wilson CI.

Returns: Dict with "sql_parse_valid" MetricResult, and optionally "sql_exec_valid" if connection is provided.