ChiuHsin

What's the meaning of TextEncoder.BERT_SPECIAL_COUNT, TextEncoder.TextEncoder.BERT_UNUSED_COUNT

4

When I use the BERT-keras, I don't understand this part: `class TextEncoder: PAD_OFFSET = 0 MSK_OFFSET = 1 BOS_OFFSET = 2 DEL_OFFSET = 3 # delimiter EOS_OFFSET = 4 SPECIAL_COUNT...

loss does not decrease during training

1

Hello, I tried to simplify your code for NER task. I made a model as below ``` python def load_model(self): self.encoder = create_transformer(embedding_layer_norm=True, neg_inf=-10000.0, use_attn_mask=self.config.use_attn_mask, vocab_size=self.bert_config.vocab_size, accurate_gelu=True, layer_norm_epsilon=1e-12, max_len=self.config.max_len, use_one_embedding_dropout=True,...

ValueError: Cannot create a tensor proto whose content is larger than 2GB.

1

hi there I am trying to start learning this code, however after i download and install everything I try to run your example code but I am getting this error,...