NER-BiLSTM-CRF-PyTorch
NER-BiLSTM-CRF-PyTorch copied to clipboard
PyTorch implementation of BiLSTM-CRF and Bi-LSTM-CNN-CRF models for named entity recognition.
NER-BiLSTM-CRF-PyTorch
PyTorch implementation of BiLSTM-CRF and Bi-LSTM-CNN-CRF models for named entity recognition.
Requirements
- Python 3
- PyTorch 1.x
Papers
- Bidirectional LSTM-CRF Models for Sequence Tagging (Huang et. al., 2015)
- the first paper apply BiLSTM-CRF to NER
- Neural Architectures for Named Entity Recognition (Lample et. al., 2016)
- introducing character-level features: pre-trained word embedding(skip-n-gram)with character-based word embeddings trained by RNN
F1: 90.94 in CoNLL 2003 (English) & 91.47 in CoNLL++
- End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF (Ma et al., 2016)
- character-level information trained by CNNs
F1: 91.21 in CoNLL 2003 (English) & 91.87 in CoNLL++
- A Deep Neural Network Model for the Task of Named Entity Recognition (Le et al., 2018)
- capitalization features
F1: 91.22 in CoNLL 2003 (English)
Dataset
- CoNLL 2003 (English)
Evaluation
- conlleval: Perl script used to calculate FB1 (phrase level)
Model
- Embeddings
- 100d pre-trained word embedding with Glove
- 25d charactor embedding trained by CNNs (Ma et al., 2016)
- BiLSTM-CRF (Lample et. al., 2016)
Results
Trained with Tesla T4 for for one night (70 epochs), obtain 91.01% F1.
Future Works
- Next papers:
- BiLSTM-CRF+ELMo ((Peters et al., 2018)
- LM-LSTM-CRF (Liu et al., 2018)
- Flair
- ...
- 中文 NER
- Batch training
References
- https://pytorch.org/tutorials/beginner/nlp/advanced_tutorial.html
- https://github.com/ZhixiuYe/NER-pytorch