latent.guardrails.metrics

Guardrail evaluation metrics compatible with latent.optimize.dspy.metrics.adapt_metric().

Functions

guardrail_adherence_metric

guardrail_adherence_metric(example: dict | Any, prediction: dict | Any, trace: Any = None) -> float

1.0 if blocked/passed matches expected_blocked, else 0.0.

Args: example: dict or DSPy Example with expected_blocked: bool prediction: dict or DSPy Prediction with blocked: bool trace: Unused (DSPy compat)

Returns: 1.0 on correct prediction, 0.0 otherwise.

per_scanner_breakdown_metric

per_scanner_breakdown_metric(rule_name: str) -> Callable

Return a metric function scoped to a specific rule.

Returns 1.0 if the named rule's fire/no-fire matches expected_blocked.

Args: rule_name: The rule name to check in violated_rules.

Returns: A metric callable compatible with adapt_metric().