biobert-pytorch
biobert-pytorch copied to clipboard
Error when running evaluation on BioASQ 4-6b datasets
@jhyuklee This happens only with datasets of 4b to 6b. the 7b dataset runs normally just like the example in the readme.the error message I think comes from converting the prediction file to a bioasq evaluation format
key error
qa['answers']
the command used to run the evaluation
python biobert-pytorch/question-answering/run_factoid.py \
--model_type bert \
--train_file ${DATA_DIR}/BioASQ-train-factoid-${TEST}b.json \
--model_name_or_path deepset/bert-base-cased-squad2 \
--do_eval \
--predict_file ${DATA_DIR}/BioASQ-test-factoid-${TEST}b-${EXPER}.json \
--golden_file ${DATA_DIR}/${TEST}B{EXPERR}_golden.json \
--per_gpu_eval_batch_size ${BATCH_SIZE} \
--max_seq_length ${MAX_LENGTH} \
--seed ${SEED} \
--official_eval_dir ${OFFICIAL_DIR} --output_dir \
${SAVE_DIR} \
--overwrite_output_dir