Compare Implementation of DML IRM in Causalis and DML IRM in DoubleMl
Compare Implementation of DML IRM in Causalis and DML IRM in DoubleMl
In Causalis we have dml_ate_source() implementation. It calls dml_irm_obj.fit() from DoubleML. Let's compare it with Causalis dml_ate()
Using DGP from Linear and Nonlinear Data Generating Process benchmarking
Treatment share ≈ 0.2052 Ground-truth ATE from the DGP: 0.800 Ground-truth ATT from the DGP: 0.800
| y | d | tenure_months | avg_sessions_week | spend_last_month | premium_user | urban_resident | |
|---|---|---|---|---|---|---|---|
| 0 | 1.903910 | 0.0 | 12.130544 | 4.056687 | 181.570607 | 0.0 | 0.0 |
| 1 | 3.388144 | 0.0 | 19.586560 | 1.671561 | 182.793598 | 0.0 | 0.0 |
| 2 | 8.456512 | 1.0 | 39.455103 | 5.452889 | 125.185708 | 1.0 | 1.0 |
| 3 | 5.535970 | 1.0 | 26.327693 | 5.051629 | 4.932905 | 0.0 | 1.0 |
| 4 | 4.965140 | 1.0 | 35.042771 | 4.933996 | 23.577407 | 0.0 | 0.0 |
ATE
Real ATE = 0.8 Estimated with Causalis = 0.7506313322797796 in (0.6668061755942251, 0.8344564889653342) Estimated with DoubleML = 0.7951243102839649 in (0.7340549132666624, 0.8561937073012674)
ATTE
Real ATTE = 0.8 Estimated with Causalis = 0.8238669785039335 in (0.7558326760362799, 0.8919012809715872) Estimated with DoubleML = 0.7951243102839649 in (0.7340549132666624, 0.8561937073012674)
Conclusion
As we see estimates with Causalis are very close to the DoubleML. Nuance functions were out of box catboost model