keras-tuner
keras-tuner copied to clipboard
keras-team
Improve Logging
SO first of all, love the idea behind this... but it seems a little lacking to my taste. It seems like it's easy enough to queue up a tonne of trials... but most of the logging/figuring out how something is performing seems so hidden.
I would use: oracle.json to at least manually look at the status of runs, but it doesn't seem to actually update. Is that expected or a bug? It only ever seems to list 2 trials.
This kills my confidence that if there is a power outage or some other issue along the way it stands any chance of actually resuming. Maybe it does a directory scan if running again... but it just seems like sooo much of what it's doing is hidden it would be nice if there was more overview details being stored in a convenient location. Like a CSV as it's scanning in the root folder.
Note: I should add part of my issue is exacerbated by the fact that it seems like the display of results is very much broken in Kaggle... but even then relying on printouts just isn't the best way to ensure info is being stored properly.
If you've tried ray[tune], it has a very nice callback logger they provide which integrates with TensorBoard for displaying results... and also has the ability to export CSV's/trial.
@Halopend Thank you for your suggestion. I will try out ray-tune. We will use this issue to track the ideas for improving the logging system.
