latent.agents.eval.scores¶

Score metadata types for Annotated Pydantic fields (FR-1.3).

Classes¶

BinaryScore¶

BinaryScore(description: str | None = None)

Marks a field as a binary (0/1) score.

ContinuousScore¶

ContinuousScore(min_value: float = 0.0, max_value: float = 1.0, description: str | None = None)

Marks a field as a continuous score within a range.

OrdinalScore¶

OrdinalScore(scale: tuple[int, ...], labels: dict[int, str] | None = None, pass_threshold: int | None = None, description: str | None = None)

Marks a field as an ordinal score with a discrete scale.

ScoredModel¶

ScoredModel()

BaseModel subclass that auto-injects {field}_rationale fields.

For every field annotated with BinaryScore, OrdinalScore, or ContinuousScore, a corresponding {field}_rationale: str = "" field is created automatically so the LLM can explain its reasoning.

Note: relies on Pydantic v2's __init_subclass__ timing (annotations are mutated before ModelMetaclass collects fields). Pinned to pydantic>=2.0.0,<3.0.0 in pyproject.toml to guard against breakage.

Functions¶

build_scoring_prompt¶

build_scoring_prompt(output_type: type[BaseModel]) -> str

Build scoring instructions from a model's score annotations.

Introspects the output_type's Annotated fields for score metadata and generates a human-readable scoring rubric.

Args: output_type: Pydantic model with Annotated score fields.

Returns: Multi-line string with scoring instructions, or "" when the model carries no BinaryScore / OrdinalScore / ContinuousScore annotation — there is no rubric to state, and a bare header over an empty list is worse than nothing in a prompt.

get_all_score_metadata¶

get_all_score_metadata(model: type[BaseModel]) -> dict[str, BinaryScore | OrdinalScore | ContinuousScore]

Extract all score metadata from a Pydantic model.

get_score_metadata¶

get_score_metadata(model: type[BaseModel], field_name: str) -> BinaryScore | OrdinalScore | ContinuousScore | None

Extract score metadata from a Pydantic model field's Annotated type.