Structured Configs with URL for torch.optim and torch.utils.data

[tkornuta-nvidia]

Summarizing, this PR introduces:

• [x] a separate confige.yamln configuration files for torch.optim
• [x] a separate configen.yaml configuration files for torch.utils.data ( #1 )
• [x] re-generated Structured Configs for modules from torch.optim
• [x] generated Structured Config for torch.utils.data.DataLoader class

This is the first try of using configen - in short: great job with this one, @omry, "meta-program or die!" ;) Please note for generation of all Structured Configs I have used modified version of configen: (Hydra PR: https://github.com/facebookresearch/hydra/pull/1082)

I assumed we will have different contributors for different modules. So to increase the modularity I have reorganized the source folder structure (source = folder containing specification of modules to be generated). Due to the current limitation of configen (hardcoded "configen.yaml" file) the proposed solution has the following directory structure:

 - source/
    - torch.optim/configen.yaml # contains specification of all torch.optim modules
    - torch.utils.data/configen.yaml # contains specification of torch.utils.data modules

[romesco]

@tkornuta-nvidia Are you envisioning we have many conf/configen-<module_specifc>.yaml files per project, rather than one monolithic config yaml per project? I'm not opposed, just wanna see if that's what you're proposing 🙂.

[omry]

as test, moved it to a different file - I assume we will have different contributors, hence different headers (e.g. copyright)

1. For now I think we should stick to a single file until we have a good reason to start splitting to config groups.
2. All contributors will use the same copyright: Facebook's. it's a requirement for all committed code (and anyone signing the CLA agrees to it).

Key 'init_config_dir' is not in struct I am not sure the configen pip package is updated, did you install from Hydra master? Take a look at the code and figure out if that needs to be there or not.

[omry]

@romesco: The CI failures are because you pushed stuff that failed CI checks. Please fix it :). We should only push stuff that passes all checks.

[omry]

also, I think we should get into the habit of always adding tests when we add new configs.

[romesco]

@romesco: The CI failures are because you pushed stuff that failed CI checks. Please fix it :). We should only push stuff that passes all checks.

Yes, sorry it was because of the reorganization of file structure. It's only a linting problem in the __init__.py within optim. Fixing that.

[tkornuta-nvidia]

@tkornuta-nvidia Are you envisioning we have many conf/configen-<module_specifc>.yaml files per project, rather than one monolithic config yaml per project? I'm not opposed, just wanna see if that's what you're proposing 🙂.

Yes, this is what I am proposing.

1. each file will be lighter, I think about having a file per pytorch module.
2. the header/licenses - I need to obey some rules, I guess other (external?) contributors might need to add something to their headers too...

[tkornuta-nvidia]

@romesco: The CI failures are because you pushed stuff that failed CI checks. Please fix it :). We should only push stuff that passes all checks.

ehehe, yeah, I know - still, quoting Shaggy" wasn't me" ;)

[omry]

@tkornuta-nvidia Are you envisioning we have many conf/configen-<module_specifc>.yaml files per project, rather than one monolithic config yaml per project? I'm not opposed, just wanna see if that's what you're proposing 🙂.

Yes, this is what I am proposing.

1. each file will be lighter, I think about having a file per pytorch module.

2. the header/licenses - I need to obey some rules, I guess other (external?) contributors might need to add something to their headers too...

I think we should use Hydra to compose configs for us much like we do otherwise. configen is powered by Hydra, we can have a config group for project and do a sweep to generate configs for all projects. There are also some questions are PyTorch version and Hydra version, which may be additional sweep dimensions eventually.