[WIP][example] Extending SC Schema for Experiments

[romesco]

This is very preliminary for the time being. I have a couple more ideas to simplify, but I want to make sure we think through this carefully (including naming conventions, folder structure, etc.).

@addisonklinke I would love to see your approach to doing this purely with dataclasses as well

First draft example can be run by:

# ensure you have hydra-core>=1.1.0.dev4 
cd examples/extend_via_sc_as_schema
python extend_via_schema.py optim=adam_exp_1

Feel free to provide any and all feedback.

[romesco]

I think there are at least 3 meaningful examples to work on: 1.) Barebones, configuring just one thing (like the optimizer) 2.) Full MNIST configuration. 3.) Training framework like Lightning or similar.

[addisonklinke]

@romesco Here's my approach for pure dataclasses. I'm not sure whether this would be considered best practice in Hydra - it sounds like typically MyDatamoduleConf would be a YAML file validated by the structured config schema instead of its own SC. My thinking was the concrete config would be better as a SC since they provide type safety

from dataclasses import dataclass
from omegaconf import MISSING


@dataclass
class DatamoduleConf:
    """Schema definition to be extended by all concrete datamodules"""
    _target_: str = MISSING
    batch_size: int = MISSING
    num_workers: int = -1

@dataclass
class MyDatamoduleConf(DatamoduleConf):
    """Projects should extend the datamodule schema above"""
    _target_: str = 'project.data.MyDatamodule'  # Required override
    batch_size: int = 100                        # Required override
    crop_dir: str = MISSING                      # New requirement added by the extension
    regions: bool = False                        # New default added by the extension