NER-BERT-CRF icon indicating copy to clipboard operation
NER-BERT-CRF copied to clipboard

Are there any plans to avoid using the special token X?

Open tossyi opened this issue 3 years ago • 0 comments

I think that the special token X was used in BERT paper v1 but not used in v2.

BERT paper v2 is written below.

5.3 Feature-based Approach with BERT
 We use the representation of the first sub-token as the input to the token-level classifier over the NER label set.

In the middle of Chapter 5.3, it is written that the label is set on the first token of the subword. Since there is no description of "X", "X" is not used at present.

The following issue also states that the model can learn patterns by learning only the first word of the subword with fine-tuning without using'X'. https://github.com/kamalkraj/BERT-NER/issues/1#issuecomment-474266192

Are there any plans to avoid using the special token X?

tossyi avatar Dec 13 '21 13:12 tossyi