DACBench icon indicating copy to clipboard operation
DACBench copied to clipboard

Include DAC-BO as a benchmark

Open benjamc opened this issue 1 year ago • 1 comments

Include the configuration of BO (SMAC) as a benchmark. Environment already exists here: https://github.com/automl/SAWEI/blob/AutoMLConf23/dacbo/env.py

  • [ ] Check if everything runs with current DACBench version
  • [ ] Check action space
  • [ ] Create instance set
  • [ ] Port SAWEI to policy format /check, run as baseline

benjamc avatar Nov 15 '23 13:11 benjamc

Thanks for opening! I think the most reasonable plan would be:

  • @TR0l1 (after SGD is through) makes a suggestion of how to unify the versions and update to the current DACBench interface (should basically be gymnasium update, I think) - this can happen on the dev branch since I think including it in the next update would be good anyway
  • @benjamc reviews the code and makes a suggestion as to search spaces and instance sets
  • I fit it those suggestions and do a final code review for the benchmark
  • @benjamc makes a separate PR for the SAWEI policy at some point

TheEimer avatar Nov 16 '23 08:11 TheEimer