BERT-pytorch icon indicating copy to clipboard operation
BERT-pytorch copied to clipboard

How to Output Embedded Word Vector

Open enze5088 opened this issue 6 years ago • 6 comments

I want to output the word vector

enze5088 avatar Jul 24 '19 02:07 enze5088

try this,

model = torch.load(your_model_file) vocab = WordVocab.load_vocab(your_vocab_file)

tokenEmb = model.state_dict()['embedding.token.weight'] segEmb = model.state_dict()['embedding.segment.weight'] posEmb = model.state_dict()['embedding.position.weight']

token_emb = tokenEmb[vocab.to_seq("word")[0]]

Vesauza avatar Jul 24 '19 02:07 Vesauza

Thank you, but I want to get the vector corresponding to each word, so I'm a little confused about the weight matrix.

enze5088 avatar Jul 24 '19 07:07 enze5088

I see. Thank you very much.

enze5088 avatar Jul 24 '19 07:07 enze5088

Is vocab.to_seq("word") [0] the index corresponding to Word? Can we just take the value of the corresponding matrix directly?

enze5088 avatar Jul 24 '19 07:07 enze5088

appear an error ModuleNotFoundError: No module named 'model.bert'

rhypowang avatar Nov 28 '19 10:11 rhypowang

How to Output Embedded Sentence Vector

rhypowang avatar Mar 12 '20 03:03 rhypowang