BERT-pytorch
BERT-pytorch copied to clipboard
Why Segment Embedding number only 3?
import torch.nn as nn
class SegmentEmbedding(nn.Embedding):
def __init__(self, embed_size=512):
super().__init__(3, embed_size, padding_idx=0)
This is the source code. First idx is padding, thus only 2 segment is supported. Why does Bert support 2 segments only?
因为一次放入2个句子,需要区分哪些token属于第一个句子,哪些token输入第二个句子, 再加上padding整好三个.