Issue with AllenNLP integration causes predict to not work (ArrayField.empty_field)

[ftyers]

trafficstars

When I try to run a clean checkout of UDify, I get the following error:

(udify-venv) fran@tlazolteotl /var/lib/home/fran/source/udify $ python predict.py udify-model.tar.gz  data/UD_Kiche-IU/quc_iu-ud-test.conllu logs/pred.conllu --eval_file logs/pred.json
Traceback (most recent call last):
  File "predict.py", line 14, in <module>
    from allennlp.models.archival import archive_model
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/models/__init__.py", line 6, in <module>
    from allennlp.models.model import Model
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/models/model.py", line 16, in <module>
    from allennlp.data import Instance, Vocabulary
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/data/__init__.py", line 1, in <module>
    from allennlp.data.dataset_readers.dataset_reader import DatasetReader
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/data/dataset_readers/__init__.py", line 10, in <module>
    from allennlp.data.dataset_readers.ccgbank import CcgBankDatasetReader
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/data/dataset_readers/ccgbank.py", line 9, in <module>
    from allennlp.data.dataset_readers.dataset_reader import DatasetReader
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/data/dataset_readers/dataset_reader.py", line 8, in <module>
    from allennlp.data.instance import Instance
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/data/instance.py", line 3, in <module>
    from allennlp.data.fields.field import DataArray, Field
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/data/fields/__init__.py", line 7, in <module>
    from allennlp.data.fields.array_field import ArrayField
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/data/fields/array_field.py", line 10, in <module>
    class ArrayField(Field[numpy.ndarray]):
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/allennlp/data/fields/array_field.py", line 50, in ArrayField
    @overrides
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/overrides/overrides.py", line 88, in overrides
    return _overrides(method, check_signature, check_at_runtime)
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/overrides/overrides.py", line 114, in _overrides
    _validate_method(method, super_class, check_signature)
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/overrides/overrides.py", line 135, in _validate_method
    ensure_signature_is_compatible(super_method, method, is_static)
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/overrides/signature.py", line 93, in ensure_signature_is_compatible
    ensure_return_type_compatibility(super_type_hints, sub_type_hints, method_name)
  File "/mnt/partuuid-46caa556-c2c4-eb47-907a-5d2092050724/var/lib/home/fran/source/udify-venv/lib/python3.7/site-packages/overrides/signature.py", line 288, in ensure_return_type_compatibility
    f"{method_name}: return type `{sub_return}` is not a `{super_return}`."
TypeError: ArrayField.empty_field: return type `None` is not a `<class 'allennlp.data.fields.field.Field'>`.

[robvanderg]

Not sure about allennlp 0.9.0, but for later versions these issues can be circumvented with downgrading the overrides package, i.e: pip3 install overrides==4.0.0

[robvanderg]

Update: tried it now with allennlp 0.9.0, and overrides 3.0.0 seems to work. I get a NaN loss error though, which seems to be stemming from the dependency labels. I now circumvented it by adding "if not torch.isnan(pred_output['loss']):" in line 133 of udify_model.py, but this is obviously suboptimal for performance.

Edit: changing mask to float_mask in line 279 of udify/model/dependency_decoder.py seems to resolve the error (not sure if this fix is correct?)

[Lguyogiro]

downgrading overrides works: pip install overrides==3.1.0

perhaps this version should be explicit in the requirements.txt file?

[andidyer]

downgrading overrides works: pip install overrides==3.1.0

perhaps this version should be explicit in the requirements.txt file?

I faced this issue too, and the quoted solution worked for me.