fastai_sequence_tagging
fastai_sequence_tagging copied to clipboard
sequence tagging for NER for ULMFiT
fastai_sequence_tagging
sequence tagging for NER for ULMFiT
data
to replicate result:
you can download the data/
folder from here, and put it in root directory.
run training
I am currently doing experiments in jupyter notebook coNLL_three_layer.ipynb
files modified from lesson10.ipynb
-
concat both forward and backward outputs from language model
W_LM = [W_forward, W_backward]
-
feeding word vectors from GloVe to a BiLSTM and get output
W_glove
-
concatenating these outputs
W = [W_glove, W_LM]
-
feeding
W
to another BiLSTM to get final result.
results
F1 score of 76.
(need to improve by fine tuning parameters, see how the toks are preprocessed, adding char embedding, adding CRF layer.
questions
-
which layer of lanuage model should be used for Sequence tagging problem
-
how to build a better language model for sequence tagging
relevant papers
Regularizing and Optimizing LSTM Language Models
deep contextualized word representations
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Semi-supervised sequence tagging with bidirectional language models