BERT-pytorch icon indicating copy to clipboard operation
BERT-pytorch copied to clipboard

Why Segment Embedding number only 3?

Open UTimeStrange opened this issue 2 years ago • 1 comments

import torch.nn as nn
class SegmentEmbedding(nn.Embedding):  
        def __init__(self, embed_size=512):  
                   super().__init__(3, embed_size, padding_idx=0)  

This is the source code. First idx is padding, thus only 2 segment is supported. Why does Bert support 2 segments only?

UTimeStrange avatar Nov 21 '23 09:11 UTimeStrange

因为一次放入2个句子,需要区分哪些token属于第一个句子,哪些token输入第二个句子, 再加上padding整好三个.

songyandong avatar Mar 22 '24 01:03 songyandong