latent.optimize.tracker¶
Thread-safe experiment tracker for prompt optimization.
Classes¶
ExperimentTracker¶
ExperimentTracker(lower_is_better: bool = False, checkpoint_path: Path | None = None, auto_checkpoint: bool = True, on_experiment: Callable[[ExperimentResult], None] | None = None)
Thread-safe tracker for optimization experiments.
Records experiment results, tracks the running best, optionally checkpoints to disk, emits MLflow nested runs, and fires callbacks.
Methods¶
ExperimentTracker.experiments¶
Return a copy of all recorded experiments.
ExperimentTracker.from_checkpoint¶
Load tracker state from a checkpoint file.
ExperimentTracker.record¶
record(label: str, score: float, error: str | None = None, metadata: dict[str, Any] | None = None, kept: bool | None = None) -> ExperimentResult
Record an experiment result. Thread-safe.
Args:
label: Human-readable label (e.g. "trial 3", "ace: round 2").
score: Numeric score for this experiment.
error: If set, this experiment is marked as failed and not kept.
metadata: Arbitrary metadata to attach.
kept: Explicit override for the kept decision. When None
(default), uses the internal _is_better comparison.
Returns: The recorded ExperimentResult.
ExperimentTracker.result¶
Build an OptimizationResult from all recorded experiments.
ExperimentTracker.running_best¶
Current best score (monotonically improving).
ExperimentTracker.save_checkpoint¶
Save checkpoint to disk.