feat: Add feature to store custom_files and config in a subdir for each experiment

[mattcamp]

trafficstars

The feature enables each training session to have it's config and custom_files stored within a subdir under experiments/.

This simplifies being able to locate the config and files that were used for a particular training session.

An additional flag is added to system.env to control the experiment name, which when set will cause the rest of the config to be loaded from $DR_DIR/experiments/<experiment_name>/run.env and $DR_DIR/experiments/<experiment_name>/custom_files/.

[larsll]

Question; does things like dr-increment-training and the -1 flag to dr-upload-model work?

[mattcamp]

Question; does things like dr-increment-training and the -1 flag to dr-upload-model work?

Yes, they work fine, although the increment-training will just update the current run.env inside the same experiment folder.

I plan to add another command to increment the whole experiment folder which will also update the DR_EXPERIMENT_NAME and increment model prefix + set pretrained prefix.