EconML
EconML copied to clipboard
py-why
Using FLAML in tune() methods
It's great to have tune() methods in some of the model classes (though sadly not all as yet). But they all seem to do tuning via grid search, which is quite wasteful. Could you possibly add an option to use FLAML instead of grid search, to search the hyperparameter space - should be quite easy if there's already a spec in the code for the parameter space of each class to be tuned.
A solution for some usecases (for now, CATE for binary treatments, extensions pending) is implemented in our auto-causality library, which searches over both individual model's parameters and across models to optimize a given score. Requests for extending the usecases it supports (via raising issues) are very welcome!
@kbattocchi as we discussed, I'm preparing a PR to have the hyperparameter search spaces available as @classmethods of the respective model. To do this, I need to import flaml.tune. Would you be happy for me to add flaml to requirements.txt, or if not, how should I do this? I could do a try-catch block inside the class that tries to import flaml.tune and only creates the classmethod if successful, but that feels kinda ugly...
I think ideally we should be providing the ranges in an optimizer-agnostic way, so that users can use FLAML if they want, but could use something like hyperopt instead if that suits their environment better.
If there is a need to use FLAML-specific functionality somewhere, then you may be able to try to import it but use the MissingModule class from econml.utilities in its place if it can't be imported - see e.g. https://github.com/microsoft/EconML/blob/ac12f54f4304c730d1501edaaeae2b96e9af4507/econml/_tree_exporter.py#L18-L28
How about I wrap the FLAML imports into MissingModule for now, and add an optimizer-agnostic spec as soon as someone raises an issue asking for it?