latent.stats.permutation

Permutation tests for small-sample comparison.

Reference: Delegates to scipy.stats.permutation_test.

Functions

permutation_test

permutation_test(scores_a: np.ndarray, scores_b: np.ndarray, statistic: Callable[[np.ndarray], float] = np.mean, n_permutations: int = 10000, seed: int | None = None) -> ComparisonResult

Paired permutation test for comparing two systems.

Appropriate for small samples (n < 50). Tests if the observed difference could have arisen by chance.

Reference: via scipy.stats.permutation_test with sign-flip on paired differences.

Args: scores_a: Scores from system A. scores_b: Scores from system B (paired, same order). statistic: Statistic to compare. n_permutations: Number of random permutations. seed: Random seed.

Returns: ComparisonResult with observed delta and p-value.