gluon-nlp
gluon-nlp copied to clipboard
Extend bert/embedding.py with other pre-trained models
We shall extend bert/embedding.py with other pre-trained models like XLNet, RoBERTa, R-XLM, ALBERT, etc. This is a good starting script where people try to leverage pre-trained model embeddings for their models for various use cases. @leezu this is somewhat aligned with the efforts for refactoring data preprocessing. I am also ok if we make copy-pastable embedding.py for other pre-trained models.
Hey, I want to give this a try! I cannot find the file "bert/embedding.py"!
@karish-grover the repository seems to have changed since then and a lot of other models seem to be in place now..