Because your hash key is your “random universe,” and those three fields control which universe you’re in.
You effectively hash:
Each piece solves a different problem.
1. experiment_id: isolate experiments + support reruns
Why include it?
-
Independence between experiments. If you didn’t use
experiment_id, the same user would always map to the same bucket across all experiments. That means:- If user is “treatment” once, they’d be “treatment” everywhere → correlated assignments → biased results when you compare across tests.
-
Stable within a single experiment. Given fixed
experiment_id, a user keeps their bucket forever (idempotent). -
Versioning / reruns. Want a fresh randomization? Use
experiment_id="exp_checkout_v2"and you get a new random split without touching code.
Rule of thumb: new logical test or new randomization → new experiment_id.
2. salt: prevent gaming + cross-system collisions
Why include it?
-
Security / anti-gaming. Without a secret salt, savvy users/devs could guess:
- “If my user_id ends with X I get treatment, so I’ll farm accounts with that pattern.”
-
Isolation from other uses of hashing. You might use SHA256 elsewhere;
saltensures the A/B assignment space is unique. -
Safe to share IDs. You can expose experiment names & variant labels in logs/BI without making it trivial to reverse or shape assignments.
Rule of thumb: keep salt server-side, same across platform; rotate only with a migration plan (rotation changes assignments).
3. layer_id: control mutual exclusivity & traffic layers
Think of layer_id as “which slot of the product UI this experiment lives in”.
Why include it?
-
Mutual exclusivity. Suppose you have many experiments on the home screen. You don’t want one user in 5 conflicting treatments.
- Put them in the same
layer_id="home_feed", then use the same hash stream to decide which single experiment (or variant) a user sees.
- Put them in the same
-
Independent surfaces. Experiments on unrelated areas (e.g.
"search_page"vs"pricing_page") should not compete:- Different
layer_id→ independent assignments, even for same user.
- Different
Rule of thumb:
- Use one layer per logical surface or exclusivity group.
- Default
"default"is fine if you don’t yet run overlapping tests.
So:
experiment_id→ “which experiment”.layer_id→ “which exclusivity sandbox”.salt→ “make the whole thing safe & non-predictable”.
Together they give you: deterministic, debuggable, replayable, non-correlated, and non-gameable assignments — i.e. what modern A/B platforms try to guarantee.
How layer_id is intended to be used in a full platform:
You define a “layer router”:
Example: layer_id="home_feed" decides which one of several experiments (or none) a user enters.
Those experiments share the same layer to enforce mutual exclusivity.
SRM
SRMResult(status=SRM DETECTED, p_value=3.744e-19, chi2=80.0000, df=1) Observed: {'control': 1200, 'treatment': 800} Expected: {'control': 1000.0, 'treatment': 1000.0} is_srm: True warning: None