EconML
EconML copied to clipboard
Could you tell me the details of the validity of confidence interval of MetaLearners?
Background
Thank you for developing such a great library.
I'm reading the document of MetaLearners of EconML.
The document states that the confidence interval is not typically valid:
due to their unrestricted flexibility, they typically do not offer valid confidence intervals, since it is not clear how arbitrary ML methods trade off bias and variance
However, this paper introduced algorithms to estimate confidence interval by bootstrap, and EconML have effect_interval method to estimate confidence interval of MetaLearners.
I also confirmed that effect_interval returns lower and upper bounds of each point.
T_learner.fit(Y, T, X=X, inference="bootstrap")
T_learner.effect_interval(X_test)
Questions
- Should we understand that the result of
effect_intervalcould be invalid? How can we interpret the invalidity? - Could you introduce several papers or documents that explain the reason why the confidence interval of MetaLearners could be invalid? (In my understanding, confidence interval estimated by bootstrap could be said to be valid)