latent.rag.tune

RAG hyperparameter grid search using retrieval-level metrics.

Evaluates different configurations (alpha, k1, b, k, thresholds) by computing recall@k against labeled relevant chunk IDs — no LLM calls needed.

Usage::

from latent.rag.tune import rag_grid_search

results = rag_grid_search(
    chroma_path="data/stable/chroma_db",
    questions=[{"query": "...", "relevant_chunk_ids": ["c1", "c2"]}],
    embedding_provider="sentence_transformer",
    embedding_model="intfloat/multilingual-e5-large",
)

Classes

TuneResult

TuneResult(config: dict[str, Any], recall_at_k: float, mrr: float, precision_at_k: float, avg_top_score: float, num_queries: int)

Result of a single RAG config evaluation.

TuneSpace

TuneSpace(alphas: list[float] = (lambda: [0.3, 0.5, 0.7, 0.8])(), k_values: list[int] = (lambda: [10, 15, 20])(), rrf_k_values: list[int] = (lambda: [30, 40, 60])(), bm25_k1: list[float] = (lambda: [1.2, 1.5, 2.0])(), bm25_b: list[float] = (lambda: [0.5, 0.75, 1.0])(), backends: list[str] = (lambda: ['hybrid', 'chroma'])(), tokenizers: list[str] = (lambda: ['default', 'hebrew'])())

Search space for RAG hyperparameter tuning.

Functions

rag_grid_search

rag_grid_search(chroma_path: str | Path, questions: list[dict[str, Any]], embedding_provider: str = 'voyage', embedding_model: str | None = None, collection_name: str = 'knowledge_base', space: TuneSpace | None = None, max_configs: int = 100) -> list[TuneResult]

Run grid search over RAG configs using retrieval-level metrics.

Args: chroma_path: Path to ChromaDB persistence directory. questions: List of dicts with "query" and "relevant_chunk_ids" keys. embedding_provider: Embedding provider name. embedding_model: Embedding model name. collection_name: ChromaDB collection name. space: Search space (defaults to standard grid). max_configs: Maximum configs to evaluate.

Returns: List of TuneResult sorted by recall@k (descending).