srm

srm

Sample Ratio Mismatch (SRM) utilities for randomized experiments.

This module provides a chi-square goodness-of-fit SRM check for randomized experiments. It accepts observed assignments as labels or aggregated counts and returns a compact result object with diagnostics.

Classes

• SRMResult – Result of a Sample Ratio Mismatch (SRM) check.

Functions

• check_srm – Check Sample Ratio Mismatch (SRM) for an RCT via a chi-square goodness-of-fit test.

SRMResult

Result of a Sample Ratio Mismatch (SRM) check.

Attributes

• chi2 (float) – The calculated chi-square statistic.
• p_value (float) – The p-value of the test, rounded to 5 decimals.
• expected (dict[Hashable, float]) – Expected counts for each variant.
• observed (dict[Hashable, int]) – Observed counts for each variant.
• alpha (float) – Significance level used for the check.
• is_srm (bool) – True if an SRM was detected (chi-square p-value < alpha), False otherwise.
• warning (str or None) – Warning message if the test assumptions might be violated (e.g., small expected counts).

alpha

chi2

expected

is_srm

observed

p_value

warning

check_srm

Check Sample Ratio Mismatch (SRM) for an RCT via a chi-square goodness-of-fit test.

Parameters

• assignments (Iterable[Hashable] or Series or CausalData or Mapping[Hashable, Number]) – Observed variant assignments. If iterable or Series, elements are labels per unit (user_id, session_id, etc.). If CausalData is provided, the treatment column is used. If a mapping is provided, it is treated as {variant: observed_count} with non-negative integer counts.
• target_allocation (dict[Hashable, Number]) – Mapping {variant: p} describing intended allocation as probabilities.
• alpha (float) – Significance level. Use strict values like 1e-3 or 1e-4 in production.
• min_expected (float) – If any expected count < min_expected, a warning is attached.
• strict_variants (bool) – - True: fail if observed variants differ from target keys.
• False: drop unknown variants and test only on declared ones.

Returns

• SRMResult – The result of the SRM check.

Raises

• ValueError – If inputs are invalid or empty.
• ImportError – If scipy is required but not installed.

Examples: