armory
armory copied to clipboard
Add support for ablation studies.
I.e. take in a collection of config files (or optimally a set of learning-rates or base-images or just a generic list of kwargs to modify) and output a single result file with each in a separate "result" object.
Are you aware of Armory's sweep attack capability? It's not the exact same thing, but maybe partially meets this need. It sounds like you are looking for a full result for each parameter value, while the sweep attack goes from weakest to strongest and I believe it skips samples if they were successfully broken with weaker parameters.