Sample imbalance between two treatments.

[Jinwoo-Yi]

trafficstars

Hi, guys! thank you for publishing a beautiful causal ML packages.

I'm going to compare causal effects of two treatments W1 and W2 to the same sample using GRF package. Before the main analysis, my colleagues advised me to check sample imbalance between W1-treated and W2-treated samples. In detailed, 49 subjects are W1-treated among total sample size 593 and 131 subjects are W2-treated among from the same data.

Covariates and outcomes are all the same between two causal tree models for treatment-W1 and treatment-W2.

Should this kind of sample imbalance be controlled in comparative research? (i.e. resampling to make the sample sizes of W1-treated and W2-treated subjects equal)

I look forward hearing from you!

Sincerely, Jinwoo

[erikcs]

Hi @Jinwoo-Yi, it sounds like with just 49 W1=1 then lack of overlap could potentially be an issue: https://grf-labs.github.io/grf/articles/diagnostics.html#assessing-overlap. Instead of fitting two forests, you could also use multi_arm_causal_forest with W a categorical vector for W0, W1, and W2. If you call average_treatment_effect on that object it will give a warning if lack of overlap seems to be an issue.