How to use BERT embedding?

Open nashid opened this issue 3 years ago • 3 comments

Is it possible to use BERT word embeddings along with this NMT implementation?

The goal is to use a pre-trained BERT language model so the contextualized embedding could be leveraged.

I am wondering whether anyone implemented or was able to run this model with any other contextualized embedding like ELMO or BERT.

Feb 23 '22 17:02 nashid

@mommi84 any idea?

Feb 23 '22 17:02 nashid

That would be a very useful feature.

Mar 23 '22 23:03 namavar-marjane

This would be very useful as contextual embedding became a norm nowadays. Wondering anyone implemented or was able to run this model with any contextualized embedding?

Mar 23 '22 23:03 nashid