latent.gates.gating¶
Threshold gating and multiple comparison correction.
Classes¶
GateSpec¶
A quality-gate threshold with per-metric strictness.
Usable anywhere a bare float threshold is accepted (analyze(gates=...),
the pre-built flows, :func:threshold_gate). A bare float keeps the default
lower_ci strictness; use GateSpec when a metric needs
point_estimate gating — e.g. small-n binary rates, where the Wilson
lower bound of even a perfect run sits well below any high threshold.
lower_is_better=True turns the gate into a ceiling: values at or below
threshold pass (inclusive — a 0.0 rate passes a 0.0 ceiling), and
lower_ci strictness checks the CI upper bound. None (the default)
pins nothing and defers to the caller's
lower_is_better/metric_directions; True or False pins the
direction, and a caller that explicitly disagrees gets a ValueError.
The tri-state is the field's own value, so it survives
model_dump()/model_validate(). Ceilings never yield a promotion
target.
Functions¶
effective_direction¶
effective_direction(threshold: float | GateSpec | None, caller_direction: bool | None, metric_name: str = '') -> bool
Resolve the one comparison direction a metric's gate and comparison share.
The shared contract between :func:threshold_gate and
:func:~latent.stats.report.analyze, which resolves it once per metric and
feeds both the gate and the comparison.
A GateSpec that pins lower_is_better wins; None on either side
defers to the other; two pinned values that disagree raise, symmetrically.
Nothing pinned anywhere means higher-is-better.
Raises: ValueError: The spec and the caller pin opposite directions.
enforce_gates¶
enforce_gates(report: StatisticalReport | Mapping[str, Any], only: Collection[str] | None = None) -> None
Raise :class:~latent.exceptions.GateFailure if any gate failed.
The single raise site shared by flow-teardown enforcement, CI scripts,
and explicit callers. Owns the smoke-run invariant: payloads stamped
sampled or gates_disabled log the skip and return, so a sampled
run can never gate no matter where enforcement is invoked from.
Args:
report: A :class:StatisticalReport, or a payload mapping as
produced by model_dump() / a published report JSON. Anything
else is rejected rather than treated as gateless — in particular
a :class:~latent.gates.publish.PublishedReport, whose dump is
the {report, markdown, payload} wrapper; pass .report.
only: When given, enforce only gates whose metric_name is in
the collection. Names matching no gate are ignored — this is a
filter, not an assertion; loud stale-key failure lives in the
teardown layer, which knows the thresholds section.
Raises:
TypeError: report is neither a report nor a mapping.
ValueError: The mapping carries no gates key, so it cannot be a
published report and "nothing failed" would be a lie.
GateFailure: At least one enforced gate has passed == False.
multiple_comparison_correction¶
Adjust p-values for multiple comparisons.
Args:
p_values: Raw (unadjusted) p-values.
method:
"holm" -- Holm-Bonferroni (default, controls FWER).
"fdr_bh" -- Benjamini-Hochberg (controls FDR).
Returns: List of adjusted p-values in the same order as the input.
Raises: ValueError: If method is not recognised.
threshold_gate¶
threshold_gate(metric: MetricResult, threshold: float | GateSpec, strictness: str = 'lower_ci', lower_is_better: bool | None = None) -> GatingResult
Check whether a metric exceeds a quality threshold.
Args:
metric: A :class:MetricResult containing the point estimate and CI.
threshold: The minimum acceptable value — a bare float, or a
:class:GateSpec carrying its own strictness. A GateSpec
takes precedence over the strictness argument, and over
lower_is_better when its direction is pinned.
strictness:
"point_estimate" -- pass if the point estimate exceeds the
threshold (lenient).
"lower_ci" -- pass if the lower bound of the CI exceeds the
threshold (stricter, default).
lower_is_better: When True, flip the comparison so that values
at or below the threshold pass (inclusive) instead of strictly
above. None (the default) pins nothing and defers to
GateSpec.lower_is_better; pinning the opposite of a pinned
spec raises ValueError.
Returns: GatingResult indicating pass / fail.
Raises:
ValueError: If strictness is not a recognised mode, or
lower_is_better and the GateSpec pin opposite directions.