auto-sklearn
auto-sklearn copied to clipboard
automl
[Question] Is there any straight forward way to retrieve the solution and prediction vector during CV?
Hello everyone!
I was wondering if there was any way to retrieve the predictions and associated solutions that were used to compute the metrics during training. Specifically, in the case of a 10 fold CV it would correspond to an array containing 10 entries with each entry having in turn the predictions and the labels of that fold.
Thank you for your time in advance!
Hi @chogovadze,
Sorry, there's no straight forward way to do this as far as I'm aware, although @mfeurer might know more. Perhaps in the future but for now a lot of the internals are hidden away and hard to access. We also try to limit memory consumption so keeping the splits in memory is not something we would do (I hope) and I imagine the splits are not written to disk either.
Best, Eddie
No, unfortunately, we don't have such information available at the moment. It would be good to have access to this kind of information in the future, though, so I'm marking it as feature.
@eddiebergman I see. I tried to circumvent this issue using global variables but as expected this does not work. @mfeurer I look forward to it! Thanks again