ImportError: cannot import name 'Dataset'

[krsnaa]

Since I was using allennlp in a different project, I continued using the pip version and not the allennlp branch - git@5fd28f0f63d8ca96fc0931bebac8224fa071c35f outlined in this repo.

When running the baseline solvers (entailment based or BiDAF), I get the error ImportError: cannot import name 'Dataset' [entailment_tuple_reader.py] - which in the current release of allennlp seems to have been refactored away from allennlp.data.dataset.

Any ETA on when the arc-solvers codebase will be compatible with the release version of allennlp?

[nijianmo]

Same issue here. I think it's because the EntailmentTupleReader is written as the previous version. Have you tried to rewrite this class?

[tusharkhot]

Since the numbers were reported with the 5fd28f0f63d8ca96fc0931bebac8224fa071c35f version of AllenNLP, we wanted to stick with the same version for reproducibility. If there is interest, we can create a new branch in this repo with the latest version of AllenNLP

[nijianmo]

Hi Tushar,

Thanks for the reply! I tried to rewrite the code using the latest version and was able to get similar results.

I have one more question about adapting bidaf model to arc. As mentioned in the paper "BiDAF was trained on SQuAD then further tuned to science questions using continued training. ", I wonder how you tune the model? For science questions, we do not have start and end position for ground truth labels. How do you handle that?

[tusharkhot]

We used the BiDAF models trained on the Science and Bio questions in a previous AAAI paper. To create training data for the RC models from multiple-choice, we identified spans with sufficient overlap with the correct answer choice in the retrieved sentences. These spans were marked as the correct answers for training the BiDAF model. Daniel, the first author on the AAAI paper, would have a better idea about the exact details