co-squac icon indicating copy to clipboard operation
co-squac copied to clipboard

The predict command in the README fails.

Open se4u opened this issue 5 years ago • 1 comments

Running the predict command fails with the error

$ python -m allennlp.run predict models/squad2.tar co-squac/datasets/converted/quac_dev.json --use-dataset-reader --output-file output/squad2.quac --batch-size 10 --cuda-device 0 --silent

 INFO - root - Loading a model trained before embedding extension was implemented; pass an explicit vocab namespace if you want to extend the vocabulary.
ERROR - allennlp.data.vocabulary - Namespace: yesno_labels 
ERROR - allennlp.data.vocabulary - Token: y                                                                                                            
  File "/home/ec2-user/.local/lib/python3.6/site-packages/allennlp/data/fields/label_field.py", line 82, in index      
    self._label_id = vocab.get_token_index(self.label, self._label_namespace)  # type: ignore                          
  File "/home/ec2-user/.local/lib/python3.6/site-packages/allennlp/data/vocabulary.py", line 637, in get_token_index   
    return self._token_to_index[namespace][self._oov_token]                                                            
KeyError: '@@UNKNOWN@@'                                                                                                

The error is probably due to the fact that the yesno_label is always x in squad, but it can be y in quac.

$ tar -axf squad2.tar vocabulary/yesno_labels.txt -O 
x

This error may only be coming up now after the update of allennlp. Could you say which allennlp version you used for your experiments?

se4u avatar Dec 29 '19 20:12 se4u

I have temporarily solved this problem by untarring the squad2.tar file to a tmp folder and manually editing the vocab files

$ cat models/tmp/vocabulary/yesno_labels.txt models/tmp/vocabulary/followup_labels.txt
x
y
n

y
m
n

It will be great if you could say if that's the right approach or not. At least the command in the README works now.

se4u avatar Dec 29 '19 20:12 se4u