BERT-pytorch
BERT-pytorch copied to clipboard
codertimo
Google AI 2018 BERT pytorch implementation
the script is similar to https://github.com/google-research/bert/blob/master/create_pretraining_data.py from google-research. it can convert a document into bert trainning data
I'm trying to add BERT as trainable part to my model and want to pass some data to it. Could you complete my code example with some x and segment_info?...
According to the code ``` def random_word(self, sentence): tokens = sentence.split() output_label = [] for i, token in enumerate(tokens): prob = random.random() if prob < 0.15: # 80% randomly change...
when the corpus is: ` how are you \ tnice to meet you` and apply `bert-vocab ` cmd, the output of the vacab is `['', '', '', '', '', 'you',...
Building the same corpus with original paper. Please share your tips to preprocess and download the file. It would be great to share preprocessed data using dropbox or google drive...
I think it's reasonable to tie the input and output embedding. Especially the output embedding along each token. But I still can't get a way to do this. Any one...
Hi there, In my endeavor to construct a BERT model from the ground up for the purpose of gaining hands-on experience and a comprehensive understanding of the model, I have...
``` import torch.nn as nn class SegmentEmbedding(nn.Embedding): def __init__(self, embed_size=512): super().__init__(3, embed_size, padding_idx=0) ``` This is the source code. First idx is padding, thus only 2 segment is supported. Why...
### What happened？ There are 1 security vulnerabilities found in torch 0.4.0 - [CVE-2022-45907](https://www.oscs1024.com/hd/CVE-2022-45907) ### What did I do？ Upgrade torch from 0.4.0 to 1.13.1 for vulnerability fix ### What...
