latent.optimize.eval

Unified evaluation contracts for latent-py optimizers.

Classes

AgentAdapter

AgentAdapter()

Minimal protocol for adapting agents to RAG eval.

EvalReport

EvalReport(primary_score: float, ci_lower: float | None = None, ci_upper: float | None = None, raw_scores: list[float] | None = None, by_category: dict[str, float] | None = None, secondary_metrics: dict[str, float] = dict(), metadata: dict[str, Any] = dict())

Standardized evaluation report returned by all optimizer eval functions.

Functions

eval_fn_from_adapter

eval_fn_from_adapter(adapter: AgentAdapter, judge: Any, query_column: str = 'query', reference_column: str = 'expected') -> RAGEvalFn

Build a RAGEvalFn from an AgentAdapter + Judge.

Methods

AgentAdapter.configure

configure(config: dict[str, Any]) -> None

AgentAdapter.invoke

invoke(query: str) -> str

Attributes

AutoResearchEvalFn

RAGEvalFn