armory
armory copied to clipboard
twosixlabs
use Hydra and Hydra-zen for configuration management and loading
https://hydra.cc/
Hydra-zen adds python things to it (pure python) https://github.com/mit-ll-responsible-ai/hydra-zen
I think it could really simplify a lot of our command line, config parsing, python object loading, and config verification.
Hydra-zen does offer some things that we have wanted for Armory.
It allows hierarchical dataclass-like initializing from declared defaults: an Experiment (config.json) can be composed of a Dataset, Attack, etc. You can also "subclass" with configuration overrides, for example, a carla_obj_det_dpatch_defended can be described as a carla_obj_det_dpatch_undefended plus a specific Defense override to what is null in the base case. Similarly, a performer could define a new Experiment as taking a dpatch_undefended base configuration and overriding just the parameters relevant to their defense.
Hydra-zen also creates detailed, datetime separated logs of how it was invoked and where parameter variants came from. This could naturally output to .armory/output_dir enriching the data output.json we currently put there.
The documentation is complete, but the learning curve is longer than I'd like because there seems to be a confusing amount of library specific boilerplate. If we use this though, it will be mostly set-and-forget, it seems.
