bi-att-flow icon indicating copy to clipboard operation
bi-att-flow copied to clipboard

Published 20 hours ago • verified publisher icon allenai

How do you get results for dev_data with squad official evaluator?

Open helloshuangzi opened this issue 5 years ago • 1 comments

Hi, How do you get results for dev_data in the readme.txt. It is mentioned the results was got with squad official evaluator. Do you need to change any code, or just use

python squad/evaluate-v1.1.py $HOME/data/squad/dev-v1.1.json out/basic/00/answer/dev-####.json

I got the dev_f1 ~44.0 when I try. There is such a large gap. Could anyone can share your practice?

Thanks, Allen.

helloshuangzi avatar Sep 06 '18 03:09 helloshuangzi

Hi any update on this?

rbhat03 avatar Mar 26 '20 05:03 rbhat03