Unconfoundedness

We call 'Uncofoundedness' a scenario where a treatment is not randomly assigned to participants, so confounders effect on treatment assignment and outcome. We have client - level data. Confounders were measured before treatment and outcome after

Let's look at the example:

In our ecosystem we have a product, which effect on LTV we want to estimate

Treatment - first purchase in product.

Outcome - LTV after first purchase.

We will test hypothesis:

$H_o$ - There is no difference in LTV between treatment and control groups.

$H_a$ - There is a difference in LTV between treatment and control groups.

Causal Assumptions

Unconfoundedness: We control all confounders

Overlap: propensity distributions overlap

SUTVA: no interference and consistent treatment definitions. By design

DML works correctly: nuisance functions are correctly specified

No leakage: confounders are measured before treatment

Data

For the analysis you need data in pandas dataframe:

• treatment column in binary format (1/0)
• outcome column numeric format, measured after treatment time
• user_id column (Optional, but useful)
• confounders columns (Optional, measured before treatment time, numeric format, used for causal assumption check)

We will take data from Causalis DGP.. Read more at https://causalis.causalcraft.com/articles/generate_obs_hte_26_rich

EDA

Our data has strong treatment class disbalance. Only 5% of sample activated in treatment.

Treatment group has lower mean LTV. It's too early to draw conclusions.

We see large right tale

We see many outliers. It's common situation for LTV metric. Dropping them will lead to a biased conclusion

As we see clients are differ on this confounders. We need to controll them to make causal inference

Inference

ATTE is right estimand here. We will estimate effect on clients that were treated, had first purchase in our product

Read more about DML at https://causalis.causalcraft.com/articles/unconfoundedness-model

Our estimate is 12.1542 dollars (ci_abs: 7.7933, 16.5152). Mean in treatment group is 58.5062 dollars, so without our product it would be 46.3520 dollars.

Refutation

Unconfoundedness

balance_max_smd is 0.011635 so dml specification dealt with controlling confounders

Sensitivity

SUTVA

SUTVA is untestable from data alone, so we call it true by design

Score

DML is specified correctly. There are many outliers in data that effect the score

Overlap

Customers are not inclined to activate the our product

if calib_ECE is YELLOW/RED look at plot_propensity_reliability(dml_result)

Logistic recalibration trend is a fitted correction curve for your predicted propensities.

The plot takes your original propensity p and fits this model:

corrected_p = sigmoid(alpha + beta * logit(p)) So the orange line answers:

If I recalibrate these predicted probabilities to better match observed treatment frequencies, what corrected probability would I use?

How to read it:

• If orange line matches the diagonal: predictions are already well calibrated
• If orange line is below the diagonal: model tends to overpredict treatment
• If orange line is above the diagonal: model tends to underpredict treatment

Rule of thumb:

• ECE near 0 and slope near 1: great
• ECE near 0 but slope far from 1: average calibration is fine, but probabilities are distorted in shape
• Many tiny bad bins + one huge good bin: visually noisy, but often not a major practical issue

Let's compare with oracle effect

Conclusion

Estimated ATTE is 9.9026 (ci_abs: 4.4731, 15.3320) dollars. The DoubleML-compatible sensitivity protocol returns PASS for all three primary benchmarks, but with a limited robustness margin: tenure_months and income_monthly use boundary calibration (cf_d_raw < 0 clipped to cf_d=0, rho=1 fallback), while the rho=1 adversarial scenario for avg_sessions_week fails.