Causal Inference Algorithms Evaluation

Project Overview

This project evaluates the performance and computational efficiency of three causal inference algorithms in computing the Average Treatment Effect (ATE). The study compares these methods across two distinct data environments: High Dimensional Data** and **Low Dimensional Data.

Methodology

Propensity Score Estimation

The foundational method used for all three algorithms is L1 penalized logistic regression. To improve the accuracy of the propensity scores, two primary steps were implemented:

1. Feature Scaling: Scaling the features prior to modeling.
2. Hyperparameter Optimization: Optimizing the hyperparameters for the regression model.

Algorithms Evaluated

• Propensity Matching + Propensity Score: Creates a series of matched sets in an optimal way so that each matched set contains at least one treated individual and at least one control individual.
• Doubly Robust Estimation: Combines an outcome regression model with weighting by a propensity score model.
• Stratification: Ranks and stratifies mutually exclusive subsets based on the propensity scores. Within each stratum, subjects have roughly similar values of the propensity scores.

Evaluation and Comparison

Algorithm	Data Type	Run Time	ATE	Accuracy
Propensity Scores Matching (full)	Low	1.7	0.36	17%
Doubly Robust Estimation	Low	0.11	2.09	100%
Stratification	Low	0.03	2.38	86%
Propensity Scores Matching (full)	High	11.81	-11.71	21%
Doubly Robust Estimation	High	0.23	-57.04	96%
Stratification	High	0.03	-59.83	91%

Conclusion

• Doubly Robust Estimation is the best performing model, returning ~100% accuracy on the low dimensional dataset and 96% accuracy on the high dimensional dataset.
• Stratification provides the least run time (highest efficiency) across all models while maintaining a relatively high accuracy.
• Propensity Score Matching resulted in the longest run time and lowest accuracy of all three methods evaluated.

Github Code Contributors:

• Fang, Zi ([email protected])
• Gao, Catherine ([email protected])
• Sang, Siyuan ([email protected])
• Washington, Eve ([email protected])