Compare Implementation of DML IRM in Causalis and DML IRM in DoubleML
Comparing IRM model from Causalis with dml.DoubleMLIRM from DoubleML with default CatboostRegressor and CatboostClassifier for g0, g1 amd m
DGP
We will use DGP: generate_obs_hte_26_rich()
read more at this notebook
Result
| y | d | tenure_months | avg_sessions_week | spend_last_month | age_years | income_monthly | prior_purchases_12m | support_tickets_90d | premium_user | mobile_user | urban_resident | referred_user | m | m_obs | tau_link | g0 | g1 | cate | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0.000000 | 0.0 | 28.814654 | 1.0 | 77.936767 | 50.234101 | 1926.698301 | 1.0 | 2.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.047970 | 0.047970 | 1.330764 | 8.137981 | 35.177086 | 27.039105 |
| 1 | 559.364158 | 1.0 | 25.913345 | 3.0 | 53.777740 | 28.115859 | 5104.271509 | 3.0 | 0.0 | 1.0 | 1.0 | 0.0 | 1.0 | 0.049695 | 0.049695 | 2.190209 | 60.459257 | 584.580685 | 524.121427 |
| 2 | 26.143003 | 1.0 | 24.969929 | 10.0 | 134.764322 | 22.907062 | 5267.938255 | 8.0 | 3.0 | 0.0 | 1.0 | 1.0 | 0.0 | 0.077087 | 0.077087 | 1.570177 | 7.712855 | 38.297992 | 30.585137 |
| 3 | 19.283585 | 1.0 | 40.655089 | 5.0 | 59.517074 | 31.970490 | 6597.327018 | 3.0 | 2.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.069481 | 0.069481 | 1.933844 | 25.386510 | 189.737828 | 164.351318 |
| 4 | 0.000000 | 1.0 | 18.560899 | 3.0 | 74.370930 | 39.237248 | 4930.009628 | 5.0 | 1.0 | 1.0 | 1.0 | 0.0 | 0.0 | 0.047097 | 0.047097 | 1.818265 | 15.359250 | 102.433597 | 87.074347 |
Result
Ground truth ATTE is 837.4043605736649
Result
CausalData(df=(100000, 13), treatment='d', outcome='y', confounders=['tenure_months', 'avg_sessions_week', 'spend_last_month', 'age_years', 'income_monthly', 'prior_purchases_12m', 'support_tickets_90d', 'premium_user', 'mobile_user', 'urban_resident', 'referred_user'])
Comparison of Inference
Causalis
Result
| estimand | coefficient | p_val | lower_ci | upper_ci | relative_diff_% | is_significant | |
|---|---|---|---|---|---|---|---|
| 0 | ATTE | 817.128619 | 0.0 | 749.253009 | 885.00423 | 893.9941 | True |
DoubleML
Result
| coef | std err | t | P>|t| | 2.5 % | 97.5 % | |
|---|---|---|---|---|---|---|
| d | 816.580926 | 34.612272 | 23.592237 | 4.630412e-123 | 748.742119 | 884.419733 |
Conclusion
Result
| estimand | coefficient | p_val | lower_ci | upper_ci | relative_diff_% | is_significant | |
|---|---|---|---|---|---|---|---|
| 0 | ATTE | 817.128619 | 0.0 | 749.253009 | 885.00423 | 893.9941 | True |
Result
| coef | std err | t | P>|t| | 2.5 % | 97.5 % | |
|---|---|---|---|---|---|---|
| d | 817.651251 | 34.638608 | 23.605199 | 3.408269e-123 | 749.760827 | 885.541676 |
Results are very close