bert-event-extraction
bert-event-extraction copied to clipboard
nlpcl-lab
head_indexes_2d是干什么用的
x是[batch_size，SEQ_LEN，768]的bert表达 有一句代码： for i in range(batch_size): x[i] = torch.index_select(x[i], 0, head_indexes_2d[i]) 请问这是在做什么？
The words are tokenized into tokens using BERT's wordpiece tokenizer. So some words will be split into more than one token, eg: lamb -> 'la', '##mb'. head_indexes_2d captures the 'head' tokens, in other words, it captures the index of 'la' only for processing for the downstream task such as TokenClassification (eg: POS tagging, NER).
x[i] = torch.index_select(x[i], 0, head_indexes_2d[i]) is used to select the index of only the head words and ignoring the subsequent 'chunks' if the word is split into more than one token.
This is how i interpret the codes. Please do correct me if I get this wrong, anyone? :D
