Can you provide the BERT features of the candidate answers?

[zhangxi1997]

Hi, thanks for your sharing. I wonder can you provide the pre-trained BERT features of the candidate answers? Thanks a lot!

[doc-doc]

Thanks for your interest. As we finetune BERT for multi-choice QA, each answer is appended to the question as a global sentence. If you want the separated candidate feature, you can split the provided global-sentence feature according to the format (1): [CLS] question [SEP] option_0 [SEP], and the token representations behind the first '[SEP]' belong to the candidate answer. You need to use BERT's buildin tokenizer to get the tokenized represetantion of the sentence (1) and find the index of ['SEP'].

[zhangxi1997]

Thanks for your quick reply. And I wonder can you provide the finetune BERT to help me extract the candidate answer features?

[doc-doc]

Hi, currently, we are not going to release this part of code, but you can refer to this file for details.

[junwenchen]

Hi,

Thanks for the explanation. Can you let me know what loss is used for fine-tuning bert?

[doc-doc]

It's softmax cross-entropy.

[doc-doc]

Hi, please find the edited code for fintuning BERT on NExT-QA here. You can also fine-tune other datasets by using the code.

[junwenchen]

Thanks a lot