allennlp.common.checks.ConfigurationError: key "dataset" is required at location "data_loader."

[RavenKiller]

Hello, I'm trying to run the training procedure allennlp build-vocab ... and allennlp train ..., but got an error:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/common/params.py", line 238, in pop
    value = self.params.pop(key)
KeyError: 'dataset'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/thor/bin/allennlp", line 8, in <module>
    sys.exit(run())
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/__main__.py", line 34, in run
    main(prog="allennlp")
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/commands/__init__.py", line 119, in main
    args.func(args)
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/commands/build_vocab.py", line 75, in build_vocab_from_args
    make_vocab_from_params(params, temp_dir)
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/training/util.py", line 468, in make_vocab_from_params
    data_loaders = data_loaders_from_params(params, serialization_dir=serialization_dir)
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/training/util.py", line 118, in data_loaders_from_params
    data_loaders["train"] = DataLoader.from_params(
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/common/from_params.py", line 589, in from_params
    return retyped_subclass.from_params(
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/common/from_params.py", line 621, in from_params
    kwargs = create_kwargs(constructor_to_inspect, cls, params, **extras)
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/common/from_params.py", line 199, in create_kwargs
    constructed_arg = pop_and_construct_arg(
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/common/from_params.py", line 303, in pop_and_construct_arg
    popped_params = params.pop(name, default) if default != _NO_DEFAULT else params.pop(name)
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/common/params.py", line 243, in pop
    raise ConfigurationError(msg)
allennlp.common.checks.ConfigurationError: key "dataset" is required at location "data_loader."

This error occurs at both build-vocab and train phase. I'm not familiar with allennlp. If I add "dataset": "alfred" into the "data_loader" field, a more confusing error occurs:

Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/thor/bin/allennlp", line 8, in <module>
    sys.exit(run())
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/__main__.py", line 34, in run
    main(prog="allennlp")
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/commands/__init__.py", line 119, in main
    args.func(args)
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/commands/build_vocab.py", line 75, in build_vocab_from_args
    make_vocab_from_params(params, temp_dir)
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/training/util.py", line 491, in make_vocab_from_params
    vocab = Vocabulary.from_params(vocab_params, instances=instances)
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/common/from_params.py", line 589, in from_params
    return retyped_subclass.from_params(
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/common/from_params.py", line 623, in from_params
    return constructor_to_call(**kwargs)  # type: ignore
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/data/vocabulary.py", line 309, in from_instances
    for instance in Tqdm.tqdm(instances, desc="building vocab"):
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/tqdm/std.py", line 1195, in __iter__
    for obj in iterable:
  File "/home/ubuntu/miniconda3/envs/thor/lib/python3.8/site-packages/allennlp/training/util.py", line 485, in <genexpr>
    for instance in data_loader.iter_instances()
TypeError: 'NoneType' object is not iterable

Is there any solution for this error?

[siifish]

Hi, fortunately you have the same question before. I encountered the same problem when running the code, did you find any solution?

[RavenKiller]

Hi, fortunately you have the same question before. I encountered the same problem when running the code, did you find any solution?

Not yet... I guess the version of allennlp and the config file in this repository are two possible reasons. But the error still occurs even if I change the allennlp version. I'm not interested in ALFRED now, so I didn't try any further.

[siifish]

Hi, fortunately you have the same question before. I encountered the same problem when running the code, did you find any solution?

Not yet... I guess the version of allennlp and the config file in this repository are two possible reasons. But the error still occurs even if I change the allennlp version. I'm not interested in ALFRED now, so I didn't try any further.

Thanks a lot! Wish you success in further research.

[aleSuglia]

Hello @RavenKiller and @siifish, thanks for your interest in EmBERT. I'm sorry that you had these issues and I'm keen to help you out.

One big and important disclaimer: unfortunately, AllenNLP has been deprecated and won't be updated anymore. Hence, this makes things harder when it comes to reproducibility.

While we investigate the source of the issue, would you mind sharing with us the allennlp version that you're using?

[siifish]

Thanks for your attention to this issue! My allennlp version is 2.4.0, which is exactly the same with version in requirements.txt. The following are the version of all packages if you need:

Package Version absl-py 1.4.0 addict 2.4.0 ai2thor 2.1.0 aiohttp 3.8.3 aiosignal 1.3.1 alabaster 0.7.12 allennlp 2.4.0 anaconda-client 1.11.0 anaconda-project 0.11.1 anyio 3.5.0 appdirs 1.4.4 argon2-cffi 21.3.0 argon2-cffi-bindings 21.2.0 arrow 1.2.2 astroid 2.11.7 astropy 4.3.1 async-timeout 4.0.2 asynctest 0.13.0 atomicwrites 1.4.0 attrs 21.4.0 Automat 20.2.0 autopep8 1.6.0 Babel 2.9.1 backcall 0.2.0 backports.functools-lru-cache 1.6.4 backports.tempfile 1.0 backports.weakref 1.0.post1 bcrypt 3.2.0 beautifulsoup4 4.11.1 binaryornot 0.4.4 bitarray 2.5.1 bkcharts 0.2 black 22.6.0 bleach 4.1.0 blis 0.7.9 bokeh 2.4.3 boto3 1.24.28 botocore 1.27.28 Bottleneck 1.3.5 brotlipy 0.7.0 cachetools 5.3.0 catalogue 2.0.8 certifi 2022.12.7 cffi 1.15.1 chardet 4.0.0 charset-normalizer 2.0.4 click 8.0.4 cloudpickle 2.0.0 clyent 1.2.2 colorama 0.4.5 colorcet 3.0.0 conda 23.1.0 conda-content-trust 0.1.3 conda-pack 0.6.0 conda-package-handling 2.0.2 conda_package_streaming 0.7.0 conda-token 0.4.0 ConfigArgParse 1.5.3 configparser 5.3.0 constantly 15.1.0 cookiecutter 1.7.3 cryptography 37.0.1 cssselect 1.1.0 cycler 0.11.0 cymem 2.0.7 Cython 0.29.32 cytoolz 0.11.0 daal4py 2021.6.0 dash 2.8.1 dash-core-components 2.0.0 dash-html-components 2.0.0 dash-table 5.0.0 dask 2021.10.0 datashader 0.14.1 datashape 0.5.4 debugpy 1.5.1 decorator 5.1.1 defusedxml 0.7.1 diff-match-patch 20200713 dill 0.3.4 distributed 2021.10.0 docker-pycreds 0.4.0 docutils 0.17.1 entrypoints 0.4 et-xmlfile 1.1.0 fastjsonschema 2.16.2 filelock 3.9.0 flake8 4.0.1 Flask 1.1.2 fonttools 4.25.0 frozenlist 1.3.3 fsspec 2022.7.1 future 0.18.2 gensim 4.1.2 gitdb 4.0.10 GitPython 3.1.30 glob2 0.7 gmpy2 2.1.2 google-auth 2.16.0 google-auth-oauthlib 0.4.6 greenlet 1.1.1 grpcio 1.51.1 h5py 2.10.0 HeapDict 1.0.1 holoviews 1.15.0 huggingface-hub 0.12.0 hvplot 0.8.0 hyperlink 21.0.0 idna 3.3 imagecodecs 2021.8.26 imageio 2.19.3 imagesize 1.4.1 importlib-metadata 3.10.0 importlib-resources 5.2.0 incremental 21.3.0 inflection 0.5.1 iniconfig 1.1.1 intake 0.6.5 intervaltree 3.1.0 ipykernel 6.15.2 ipython 7.31.1 ipython-genutils 0.2.0 ipywidgets 7.6.5 isort 5.9.3 itemadapter 0.3.0 itemloaders 1.0.4 itsdangerous 2.0.1 jdcal 1.4.1 jedi 0.18.1 jeepney 0.7.1 jellyfish 0.9.0 Jinja2 2.11.3 jinja2-time 0.2.0 jmespath 0.10.0 joblib 1.1.0 json5 0.9.6 jsonlines 3.1.0 jsonnet 0.19.1 jsonschema 4.16.0 jupyter 1.0.0 jupyter_client 7.3.5 jupyter-console 6.4.3 jupyter_core 4.11.1 jupyter-server 1.18.1 jupyterlab 3.4.4 jupyterlab-pygments 0.1.2 jupyterlab-server 2.10.3 jupyterlab-widgets 1.0.0 keyring 23.4.0 kiwisolver 1.4.2 lazy-object-proxy 1.6.0 libarchive-c 2.9 llvmlite 0.38.0 lmdb 1.4.0 locket 1.0.0 lxml 4.9.1 Markdown 3.3.4 MarkupSafe 2.0.1 matplotlib 3.5.2 matplotlib-inline 0.1.6 mccabe 0.7.0 mistune 0.8.4 mkl-fft 1.3.1 mkl-random 1.2.2 mkl-service 2.4.0 mock 4.0.3 more-itertools 9.0.0 mpmath 1.2.1 msgpack 1.0.3 multidict 6.0.4 multipledispatch 0.6.0 munkres 1.1.4 murmurhash 1.0.9 mypy-extensions 0.4.3 nbclassic 0.3.5 nbclient 0.5.13 nbconvert 6.4.4 nbformat 5.5.0 nest-asyncio 1.5.5 networkx 2.6.3 nltk 3.7 nose 1.3.7 notebook 6.4.12 numba 0.55.1 numexpr 2.8.3 numpy 1.21.5 numpydoc 1.4.0 nvidia-cublas-cu11 11.10.3.66 nvidia-cuda-nvrtc-cu11 11.7.99 nvidia-cuda-runtime-cu11 11.7.99 nvidia-cudnn-cu11 8.5.0.96 oauthlib 3.2.2 olefile 0.46 open3d 0.16.0 opencv-python 4.7.0.68 openpyxl 3.0.10 overrides 3.1.0 packaging 21.3 pandas 1.3.5 pandocfilters 1.5.0 panel 0.13.1 param 1.12.0 parsel 1.6.0 parso 0.8.3 partd 1.2.0 pathspec 0.9.0 pathtools 0.1.2 pathy 0.10.1 patsy 0.5.2 pep8 1.7.1 pexpect 4.8.0 pickleshare 0.7.5 Pillow 9.2.0 pip 22.2.2 pkginfo 1.8.2 pkgutil_resolve_name 1.3.10 platformdirs 2.5.2 plotly 5.9.0 pluggy 1.0.0 ply 3.11 poyo 0.5.0 preshed 3.0.8 progressbar2 4.2.0 prometheus-client 0.14.1 promise 2.3 prompt-toolkit 3.0.20 Protego 0.1.16 protobuf 3.20.1 psutil 5.9.0 ptyprocess 0.7.0 py 1.11.0 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycocotools 2.0.0 pycodestyle 2.8.0 pycosat 0.6.3 pycparser 2.21 pycrypto 2.6.1 pyct 0.4.8 pycurl 7.45.1 pydantic 1.7.4 pyDeprecate 0.3.2 PyDispatcher 2.0.5 pydocstyle 6.1.1 pyerfa 2.0.0 pyflakes 2.4.0 Pygments 2.11.2 PyHamcrest 2.0.2 pylint 2.14.5 pyls-spyder 0.4.0 pyodbc 4.0.34 pyOpenSSL 22.0.0 pyparsing 3.0.9 PyQt5-sip 12.11.0 pyquaternion 0.9.9 pyrsistent 0.18.0 PySocks 1.7.1 pytest 7.1.2 python-dateutil 2.8.2 python-lsp-black 1.2.1 python-lsp-jsonrpc 1.0.0 python-lsp-server 1.5.0 python-slugify 5.0.2 python-snappy 0.6.0 python-utils 3.5.2 pytorch-lightning 1.6.0 pytz 2022.1 pyviz-comms 2.0.2 PyWavelets 1.3.0 pyxdg 0.27 PyYAML 6.0 pyzmq 23.2.0 QDarkStyle 3.0.2 qstylizer 0.1.10 QtAwesome 1.0.3 qtconsole 5.3.2 QtPy 2.2.0 queuelib 1.5.0 regex 2022.7.9 requests 2.28.1 requests-file 1.5.1 requests-oauthlib 1.3.1 rope 0.22.0 rsa 4.9 Rtree 0.9.7 ruamel.yaml 0.17.21 ruamel.yaml.clib 0.2.6 ruamel-yaml-conda 0.15.100 s3transfer 0.6.0 sacremoses 0.0.53 scikit-image 0.19.2 scikit-learn 1.0.2 scikit-learn-intelex 2021.20221004.171321 scipy 1.7.3 Scrapy 2.6.2 seaborn 0.11.2 SecretStorage 3.3.1 Send2Trash 1.8.0 sentencepiece 0.1.97 sentry-sdk 1.15.0 service-identity 18.1.0 setuptools 63.4.1 shortuuid 1.0.11 sip 6.6.2 six 1.16.0 smart-open 5.2.1 smmap 5.0.0 sniffio 1.2.0 snowballstemmer 2.2.0 sortedcollections 2.1.0 sortedcontainers 2.4.0 soupsieve 2.3.1 spacy 3.0.3 spacy-legacy 3.0.12 Sphinx 4.2.0 sphinxcontrib-applehelp 1.0.2 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.5 spyder 5.3.3 spyder-kernels 2.3.3 SQLAlchemy 1.4.39 srsly 2.4.5 statsmodels 0.13.2 subprocess32 3.5.4 sympy 1.10.1 tables 3.6.1 tabulate 0.8.10 TBB 0.2 tblib 1.7.0 tenacity 8.0.1 tensorboard 2.11.2 tensorboard-data-server 0.6.1 tensorboard-plugin-wit 1.8.1 tensorboardX 2.5.1 terminado 0.13.1 testpath 0.6.0 text-unidecode 1.3 textdistance 4.2.1 thinc 8.0.17 threadpoolctl 2.2.0 three-merge 0.1.1 tifffile 2021.7.2 tinycss 0.4 tldextract 3.2.0 tokenizers 0.10.3 toml 0.10.2 tomli 2.0.1 tomlkit 0.11.1 toolz 0.11.2 torch 1.13.1 torchaudio 0.13.1 torchmetrics 0.6.2 torchvision 0.14.1 tornado 6.2 tqdm 4.64.1 traitlets 5.1.1 transformers 4.10.0 Twisted 22.2.0 typed-ast 1.4.3 typer 0.3.2 typing_extensions 4.3.0 ujson 5.4.0 Unidecode 1.2.0 urllib3 1.26.11 w3lib 1.21.0 wandb 0.10.33 wasabi 0.10.1 watchdog 2.1.6 wcwidth 0.2.5 webencodings 0.5.1 websocket-client 0.58.0 Werkzeug 2.0.3 whatthepatch 1.0.2 wheel 0.37.1 widgetsnbextension 3.5.2 wrapt 1.14.1 wurlitzer 3.0.2 xarray 0.20.1 xlrd 2.0.1 XlsxWriter 3.0.3 yapf 0.31.0 yarl 1.8.2 zict 2.1.0 zipp 3.8.0 zope.interface 5.4.0 zstandard 0.18.0