fairseq
fairseq copied to clipboard
facebookresearch
Unable to train nlp with base_text_only_task
🐛 Bug
Following the documentation I'm trying to get NLP to work on my local machine (running on Fedora 37)
To Reproduce
$ python3.10 fairseq_cli/hydra_train.py -m --config-dir examples/data2vec/config/v2 --config-name base_text_only_task task.data=/home/my-user/TheVault/Codes/experiments/data/nlp/nlp_base.pt
[2022-12-26 10:35:02,704][HYDRA] Launching 1 jobs locally
[2022-12-26 10:35:02,704][HYDRA] #0 : task.data=/home/my-user/TheVault/Codes/experiments/data/nlp/nlp_base.pt
Traceback (most recent call last):
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/hydra/_internal/utils.py", line 198, in run_and_report
return func()
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/hydra/_internal/utils.py", line 355, in <lambda>
lambda: hydra.multirun(
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/hydra/_internal/hydra.py", line 136, in multirun
return sweeper.sweep(arguments=task_overrides)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/hydra/_internal/core_plugins/basic_sweeper.py", line 154, in sweep
results = self.launcher.launch(batch, initial_job_idx=initial_job_idx)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/hydra/_internal/core_plugins/basic_launcher.py", line 76, in launch
ret = run_job(
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/hydra/core/utils.py", line 129, in run_job
ret.return_value = task_function(task_cfg)
File "/home/my-user/TheVault/Codes/experiments/ai/fairseq/fairseq_cli/hydra_train.py", line 27, in hydra_main
_hydra_main(cfg)
File "/home/my-user/TheVault/Codes/experiments/ai/fairseq/fairseq_cli/hydra_train.py", line 31, in _hydra_main
add_defaults(cfg)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/fairseq/dataclass/initialize.py", line 61, in add_defaults
cfg[k] = merge_with_parent(dc, field_cfg)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/fairseq/dataclass/utils.py", line 500, in merge_with_parent
merged_cfg = OmegaConf.merge(dc, cfg)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/omegaconf.py", line 321, in merge
target.merge_with(*others[1:])
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/basecontainer.py", line 331, in merge_with
self._format_and_raise(key=None, value=None, cause=e)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/base.py", line 95, in _format_and_raise
format_and_raise(
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/_utils.py", line 629, in format_and_raise
_raise(ex, cause)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/_utils.py", line 610, in _raise
raise ex # set end OC_CAUSE=1 for full backtrace
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/basecontainer.py", line 329, in merge_with
self._merge_with(*others)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/basecontainer.py", line 347, in _merge_with
BaseContainer._map_merge(self, other)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/basecontainer.py", line 314, in _map_merge
dest[key] = src._get_node(key)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/dictconfig.py", line 258, in __setitem__
self._format_and_raise(
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/base.py", line 95, in _format_and_raise
format_and_raise(
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/_utils.py", line 629, in format_and_raise
_raise(ex, cause)
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/omegaconf/_utils.py", line 610, in _raise
raise ex # set end OC_CAUSE=1 for full backtrace
omegaconf.errors.ConfigKeyError: Key 'include_index' not in 'MaskedLMConfig'
full_key: include_index
reference_type=Optional[MaskedLMConfig]
object_type=MaskedLMConfig
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/my-user/TheVault/Codes/experiments/ai/fairseq/fairseq_cli/hydra_train.py", line 91, in <module>
cli_main()
File "/home/my-user/TheVault/Codes/experiments/ai/fairseq/fairseq_cli/hydra_train.py", line 87, in cli_main
hydra_main()
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/hydra/main.py", line 32, in decorated_main
_run_hydra(
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/hydra/_internal/utils.py", line 354, in _run_hydra
run_and_report(
File "/home/my-user/TheVault/Codes/experiments/ai/.venv-310/lib64/python3.10/site-packages/hydra/_internal/utils.py", line 267, in run_and_report
print_exception(etype=None, value=ex, tb=final_tb) # type: ignore
TypeError: print_exception() got an unexpected keyword argument 'etype'
Code sample
Expected behavior
To start the training for NLP.
Environment
- fairseq Version (e.g., 1.0 or main): fairseq==0.12.2
- PyTorch Version (e.g., 1.0) torch==1.13.1
- OS (e.g., Linux): Fedora 37
- How you installed fairseq (
pip, source): pip3 - Build command you used (if compiling from source): N/A
- Python version: Python 3.10.9
- CUDA/cuDNN version:N/A
- GPU models and configuration: N/A
- Any other relevant information:
$ pip3 freeze
antlr4-python3-runtime==4.8
bitarray==2.6.1
cffi==1.15.1
colorama==0.4.6
Cython==0.29.32
fairseq==0.12.2
hydra-core==1.0.7
lxml==4.9.2
numpy==1.24.0
nvidia-cublas-cu11==11.10.3.66
nvidia-cuda-nvrtc-cu11==11.7.99
nvidia-cuda-runtime-cu11==11.7.99
nvidia-cudnn-cu11==8.5.0.96
omegaconf==2.0.6
portalocker==2.6.0
protobuf==3.20.1
pycparser==2.21
PyYAML==6.0
regex==2022.10.31
sacrebleu==2.3.1
tabulate==0.9.0
tensorboardX==2.5.1
torch==1.13.1
torchaudio==0.13.1
tqdm==4.64.1
typing_extensions==4.4.0
Additional context
N/A
