Issue with hydra - Error merging data/dataset=cstm Key 'defaults' not in 'FairseqConfig'

[aprzez]

Hello,

I left the below comment under issue #3 but since it's closed I am not sure the comment will be seen.

I am trying to run the training on a custom dataset and also experiencing this issue. The file cstm.yaml is placed in here auto_avsr/conf/data/dataset and looks like this: `defaults:

self root: "/content/drive/MyDrive/sepedi/data/preprocess_datasets" label_dir: "labels" train_file: "train_labels.csv" val_file: "val_labels.csv" test_file: "test_labels.csv"`

As suggested above, I tried renaming the conf/config.yaml file. However, when I run:

!python main.py exp_dir=exp \ exp_name=trainaudio \ data.modality=audio \ ckpt_path='content/drive/MyDrive/LRS3_A_WER1.0/model.pth' \ +data/dataset=cstm \ trainer.num_nodes=1

I get:

Error merging data/dataset=cstm Key 'defaults' not in 'FairseqConfig' full_key: defaults reference_type=Optional[FairseqConfig] object_type=FairseqConfig

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

When running:

%env HYDRA_FULL_ERROR=1 !python main.py exp_dir=exp \ exp_name=trainaudio \ data.modality=audio \ ckpt_path='content/drive/MyDrive/LRS3_A_WER1.0/model.pth' \ +data/dataset=cstm \ trainer.num_nodes=1

I get: Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/config_loader_impl.py", line 720, in _merge_config ret = OmegaConf.merge(cfg, loaded_cfg) File "/usr/local/lib/python3.10/dist-packages/omegaconf/omegaconf.py", line 321, in merge target.merge_with(*others[1:]) File "/usr/local/lib/python3.10/dist-packages/omegaconf/basecontainer.py", line 331, in merge_with self._format_and_raise(key=None, value=None, cause=e) File "/usr/local/lib/python3.10/dist-packages/omegaconf/base.py", line 95, in _format_and_raise format_and_raise( File "/usr/local/lib/python3.10/dist-packages/omegaconf/_utils.py", line 629, in format_and_raise _raise(ex, cause) File "/usr/local/lib/python3.10/dist-packages/omegaconf/_utils.py", line 610, in _raise raise ex # set end OC_CAUSE=1 for full backtrace File "/usr/local/lib/python3.10/dist-packages/omegaconf/basecontainer.py", line 329, in merge_with self._merge_with(*others) File "/usr/local/lib/python3.10/dist-packages/omegaconf/basecontainer.py", line 347, in _merge_with BaseContainer._map_merge(self, other) File "/usr/local/lib/python3.10/dist-packages/omegaconf/basecontainer.py", line 314, in _map_merge dest[key] = src._get_node(key) File "/usr/local/lib/python3.10/dist-packages/omegaconf/dictconfig.py", line 258, in setitem self._format_and_raise( File "/usr/local/lib/python3.10/dist-packages/omegaconf/base.py", line 95, in _format_and_raise format_and_raise( File "/usr/local/lib/python3.10/dist-packages/omegaconf/_utils.py", line 629, in format_and_raise _raise(ex, cause) File "/usr/local/lib/python3.10/dist-packages/omegaconf/_utils.py", line 610, in _raise raise ex # set end OC_CAUSE=1 for full backtrace omegaconf.errors.ConfigKeyError: Key 'defaults' not in 'FairseqConfig' full_key: defaults reference_type=Optional[FairseqConfig] object_type=FairseqConfig

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/content/auto_avsr/main.py", line 74, in main() File "/usr/local/lib/python3.10/dist-packages/hydra/main.py", line 32, in decorated_main _run_hydra( File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/utils.py", line 346, in _run_hydra run_and_report( File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/utils.py", line 201, in run_and_report raise ex File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/utils.py", line 198, in run_and_report return func() File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/utils.py", line 347, in lambda: hydra.run( File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/hydra.py", line 100, in run cfg = self.compose_config( File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/hydra.py", line 507, in compose_config cfg = self.config_loader.load_configuration( File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/config_loader_impl.py", line 151, in load_configuration return self._load_configuration( File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/config_loader_impl.py", line 256, in _load_configuration cfg = self._merge_defaults_into_config( File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/config_loader_impl.py", line 805, in _merge_defaults_into_config hydra_cfg = merge_defaults_list_into_config(hydra_cfg, user_list) File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/config_loader_impl.py", line 777, in merge_defaults_list_into_config merged_cfg = self._merge_config( File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/config_loader_impl.py", line 724, in _merge_config raise ConfigCompositionException( hydra.errors.ConfigCompositionException: Error merging data/dataset=cstm

I am running this on colab due to issues installing fairseq editable locally.

Thank you in advance!