CAVE icon indicating copy to clipboard operation
CAVE copied to clipboard

Auto-PyTorch Integration

Open shukon opened this issue 5 years ago • 0 comments

To fully integrate APT in CAVE, APT first needs to settle for a longterm output format. At the time of writing, there is no such format, just a number of json-files that look different for different experiments. However, a uniform format as well as tensorboard logging for individual incumbents will yield a great way to compare the incumbents within tensorboard. CAVE reads APT in general from 1.4.0, assuming APT builds on HpBandSter for the basic results (configs.json and results.json). Users will have to manually write out the configspace.json as well, since that is not done within APT. If there are tensorboard-files present, they will be consumed and made available in a jupyter-notebook, see the APT-example notebook. Further integration can happen in a few places... if there are additional files in APT's output, the conversion to SMAC3 needs to be adapted. APT-specific analyzers can live in analyzer/apt/. Check out the Contribution guide for more info on adding new analyzer's.

shukon avatar Jul 01 '20 14:07 shukon