causaltune icon indicating copy to clipboard operation
causaltune copied to clipboard

best_config should return component models, too

Open TimoFlesch opened this issue 2 years ago • 3 comments

In our presentation last week it occured to me that the best_config dict seems to contain only the name of the causal model and - if it has any - its best-fitting hyperparams. For full reproducibility, it would be great if that method also returned the component models with respective hyperparams.

TimoFlesch avatar Apr 20 '22 08:04 TimoFlesch

that is already there in the .scores property which has the fullest information by estimator type, whereas things like best_config are convenience functions to extract smaller, handier subsets of that info Happy to discuss if you disagree :)

EgorKraevTransferwise avatar Apr 21 '22 10:04 EgorKraevTransferwise

Thanks for the quick reply! I see. Maybe it's a matter of semantics :) My understanding was that best_config should return the whole config of the best-fitting model logged in .scores, i.e. both the results from the outer and the inner loop, so that these could be easily passed on as arguments to a DoWhy model (or used as starting points for further fitting or whatever). Maybe an alternative would be to divide this intobest_causal_config and best_component_config but this may just confuse the user as they two are inherently linked?

TimoFlesch avatar Apr 22 '22 08:04 TimoFlesch

Hmm fair point. I think the best time to decide this will be when we actually start trying to implement that usecase, or when someone asks for it - it's generally hard to guess what users will find intuitive, or useful, unless they either tell you or you are one of the users :)

EgorKraevTransferwise avatar Apr 22 '22 09:04 EgorKraevTransferwise