barracuda.validation

Typed scenarios, deterministic seeds, parameter-recovery tables, coverage, boundary summaries, posterior comparison probabilities, and complete single-dataset validation runners.

Reusable validation and posterior-recovery utilities.

The functions in this module are deliberately independent of the web UI and filesystem layout used by the research scripts. Validation runs use the same public simulators and inference entry points as the rest of :mod:barracuda.

Attributes

COUNT_MODEL_KEYS module-attribute

COUNT_MODEL_KEYS: Final[tuple[str, ...]] = ('homo', 'z2p', 'dis2p', 'hetero3')

TRAJECTORY_MODEL_KEYS module-attribute

TRAJECTORY_MODEL_KEYS: Final[tuple[str, ...]] = tuple(trajectories.TRAJECTORY_MODEL_SPECS)

COUNT_SCENARIOS module-attribute

COUNT_SCENARIOS: Final[tuple[EventCountScenario, ...]] = (EventCountScenario('No1', 'No1: sigma_lambda=3, p_zero=0.2', 4.0, 3.0, 0.2, 'hetero3', 1), EventCountScenario('No2', 'No2: sigma_lambda=0, p_zero=0.2', 4.0, 0.0, 0.2, 'z2p', 2), EventCountScenario('No3', 'No3: sigma_lambda=3, p_zero=0', 4.0, 3.0, 0.0, 'dis2p', 3), EventCountScenario('No4', 'No4: sigma_lambda=0, p_zero=0', 4.0, 0.0, 0.0, 'homo', 4))

TRAJECTORY_SCENARIOS module-attribute

TRAJECTORY_SCENARIOS: Final[tuple[TrajectoryScenario, ...]] = (TrajectoryScenario('No1', 'No1: sigma_eta=1, beta=(0.8,-0.8)', 4.0, 2.0, 0.25, 1.0, 0.8, -0.8, 'heterogeneous_history_dependent', 1), TrajectoryScenario('No2', 'No2: sigma_eta=1, beta=(0,0)', 4.0, 2.0, 0.25, 1.0, 0.0, 0.0, 'heterogeneous_history_independent', 2), TrajectoryScenario('No3', 'No3: sigma_eta=0, beta=(0.8,-0.8)', 4.0, 2.0, 0.25, 0.0, 0.8, -0.8, 'homogeneous_history_dependent', 3), TrajectoryScenario('No4', 'No4: sigma_eta=0, beta=(0,0)', 4.0, 2.0, 0.25, 0.0, 0.0, 0.0, 'homogeneous_history_independent', 4))

_RECOVERY_COLUMNS module-attribute

_RECOVERY_COLUMNS: Final[list[str]] = ['condition', 'model_key', 'parameter', 'posterior_variable', 'truth', 'mean', 'median', 'sd', 'hdi_lower', 'hdi_upper', 'hdi_probability', 'error', 'absolute_error', 'relative_error', 'covered', 'n_draws']

_COUNT_PARAMETER_MAPS module-attribute

_COUNT_PARAMETER_MAPS: Final[dict[str, dict[str, str]]] = {'homo': {'mu_lambda': 'lambda'}, 'z2p': {'mu_lambda': 'lambda', 'p_zero': 'p_zero'}, 'dis2p': {'mu_lambda': 'mu_lambda', 'sigma_lambda': 'sigma_lambda'}, 'hetero3': {'mu_lambda': 'mu_lambda', 'sigma_lambda': 'sigma_lambda', 'p_zero': 'p_zero'}}

_DEFAULT_BOUNDARIES module-attribute

_DEFAULT_BOUNDARIES: Final[dict[str, float]] = {'sigma_lambda': 0.0, 'p_zero': 0.0, 'sigma_eta': 0.0, 'beta_f': 0.0, 'beta_s': 0.0}

simulate_event_count_data module-attribute

simulate_event_count_data = event_counts.simulate_event_counts

fit_event_count_models module-attribute

fit_event_count_models = event_counts.run_count_models

simulate_trajectory_data module-attribute

simulate_trajectory_data = trajectories.simulate_trajectory_frame

fit_trajectory_models module-attribute

fit_trajectory_models = trajectories.run_trajectory_conditions

__all__ module-attribute

__all__ = ['COUNT_MODEL_KEYS', 'COUNT_SCENARIOS', 'TRAJECTORY_MODEL_KEYS', 'TRAJECTORY_SCENARIOS', 'EventCountScenario', 'EventCountValidationResult', 'PosteriorProbabilityResult', 'TrajectoryScenario', 'TrajectoryValidationResult', 'boundary_recovery_summary', 'coverage_summary', 'event_count_recovery_table', 'fit_event_count_models', 'fit_trajectory_models', 'posterior_recovery_table', 'posterior_rope_probabilities', 'posterior_superiority_probability', 'run_event_count_validation', 'run_trajectory_validation', 'simulate_event_count_data', 'simulate_trajectory_data', 'stable_seed', 'trajectory_recovery_table']

Classes

EventCountScenario dataclass

EventCountScenario(scenario: str, label: str, mu_lambda: float, sigma_lambda: float, p_zero: float, true_model: str, seed_offset: int = 0)

Typed ground truth for one canonical event-count experiment.

TrajectoryScenario dataclass

TrajectoryScenario(scenario: str, label: str, mu_lambda: float, sigma_lambda: float, p0: float, sigma_eta: float, beta_f: float, beta_s: float, true_model: str, seed_offset: int = 0)

Typed ground truth for one canonical contact-trajectory experiment.

Attributes

mu_eta property

mu_eta: float

Logit-scale population killing propensity implied by p0.

EventCountValidationResult dataclass

EventCountValidationResult(scenario: EventCountScenario, replicate: int, simulation_seed: int, inference_seed: int, frame: DataFrame, truth: Mapping[str, Any], fits: Mapping[str, Any], evidence: DataFrame, recovery: DataFrame)

Complete in-memory result for one simulated event-count validation.

TrajectoryValidationResult dataclass

TrajectoryValidationResult(scenario: TrajectoryScenario, replicate: int, simulation_seed: int, inference_seed: int, frame: DataFrame, truth: Mapping[str, Any], fits: Mapping[str, Any], evidence: DataFrame, recovery: DataFrame)

Complete in-memory result for one simulated trajectory validation.

PosteriorProbabilityResult dataclass

PosteriorProbabilityResult(rope_lower: float, rope_upper: float, probability_below: float, probability_in_rope: float, probability_above: float, n_first: int, n_second: int, n_pairs: int)

Exact cross-draw probabilities for first - second and a ROPE.

Attributes

probability_first_superior property

probability_first_superior: float

Probability that first - second is above the ROPE.

probability_second_superior property

probability_second_superior: float

Probability that first - second is below the ROPE.

Methods:

as_dict

as_dict() -> dict[str, int | float]

Return a serialization-friendly representation.

Functions:

_finite

_finite(value: Any, name: str) -> float

_positive_int

_positive_int(value: Any, name: str) -> int

_base_seed

_base_seed(value: Any) -> int

_label

_label(value: Any, name: str) -> str

_count_truth_model

_count_truth_model(sigma_lambda: float, p_zero: float) -> str

_trajectory_truth_model

_trajectory_truth_model(sigma_eta: float, beta_f: float, beta_s: float) -> str

_json_default

_json_default(value: Any) -> Any

stable_seed

stable_seed(*parts: Any, namespace: str = 'barracuda') -> int

Derive a reproducible non-zero uint32 seed from structured values.

Unlike Python's built-in hash, this value is stable across processes. Mappings are JSON encoded with sorted keys, so their insertion order does not affect the seed.

_posterior_values

_posterior_values(idata: Any, variable: str) -> ndarray

posterior_recovery_table

posterior_recovery_table(idata: Any, truth: Mapping[str, Any], *, parameters: Sequence[str] | None = None, parameter_map: Mapping[str, str] | None = None, model_key: str | None = None, condition: str | None = None, hdi_prob: float = 0.95) -> DataFrame

Compare scalar posterior parameters with their generating truths.

parameter_map maps public/truth names to posterior variable names. The returned error is posterior mean - truth; covered states whether the closed HDI contains the truth.

event_count_recovery_table

event_count_recovery_table(results: Mapping[str, Any], truth: Mapping[str, Any], *, hdi_prob: float = 0.95) -> DataFrame

Build recovery rows for every fitted event-count model.

trajectory_recovery_table

trajectory_recovery_table(results: Mapping[str, Any], truth: Mapping[str, Any], *, condition: str | None = None, hdi_prob: float = 0.95) -> DataFrame

Build recovery rows for every fitted trajectory decision model.

_validated_group_columns

_validated_group_columns(frame: DataFrame, group_by: Sequence[str]) -> list[str]

coverage_summary

coverage_summary(recovery: DataFrame, *, group_by: Sequence[str] = ('parameter',)) -> DataFrame

Aggregate HDI coverage, bias, RMSE, and interval width.

boundary_recovery_summary

boundary_recovery_summary(recovery: DataFrame, *, boundaries: Mapping[str, float] | None = None, truth_tolerance: float = 1e-12, estimate_tolerance: float = 0.1, group_by: Sequence[str] = ('model_key', 'parameter')) -> DataFrame

Summarise recovery rows whose generating truth lies on a boundary.

boundary_coverage_rate records whether the posterior HDI includes the boundary. estimate_within_tolerance_rate uses the posterior mean and the absolute estimate_tolerance supplied by the caller.

_finite_draws

_finite_draws(values: Any, name: str) -> ndarray

posterior_superiority_probability

posterior_superiority_probability(first: Any, second: Any, *, margin: float = 0.0) -> float

Return exact P(first - second > margin) over all cross-draw pairs.

The samples are treated as independent posterior populations. Sorting and binary search avoid constructing the potentially huge Cartesian product.

posterior_rope_probabilities

posterior_rope_probabilities(first: Any, second: Any, *, rope: tuple[float, float] = (-0.1, 0.1)) -> PosteriorProbabilityResult

Return exact probabilities below, within, and above a difference ROPE.

The ROPE is closed: lower <= first - second <= upper. The below and above events are strict, so the three returned probabilities partition all cross-draw pairs exactly, including ties.

_selected_models

_selected_models(requested: Sequence[str] | str | None, available: Sequence[str]) -> tuple[str, ...]

_single_evidence_table

_single_evidence_table(fits: Mapping[str, Any], *, true_model: str) -> DataFrame

run_event_count_validation

run_event_count_validation(scenario: EventCountScenario, n_cells: int, *, observation_time: float = 1.0, replicate: int = 1, base_seed: int = 2026, settings: InferenceSettings | None = None, model_keys: Sequence[str] | str | None = None, hdi_prob: float = 0.95, progress_callback: Any = None) -> EventCountValidationResult

Simulate, fit, and assess one event-count validation dataset.

run_trajectory_validation

run_trajectory_validation(scenario: TrajectoryScenario, n_cells: int, *, observation_time: float = 1.0, replicate: int = 1, base_seed: int = 2026, settings: TrajectorySettings | None = None, model_keys: Sequence[str] | str | None = None, hdi_prob: float = 0.95, progress_callback: Any = None) -> TrajectoryValidationResult

Simulate, fit, and assess one trajectory validation dataset.