mongolian-text-classification
mongolian-text-classification copied to clipboard
Cyrillic Mongolian text classification with tensorflow 2, and also some fine-tuning on TugsTugi's Mongolian BERT model and other NLP experiments are included.
mongolian-text-classification
Mongolian cyrillic text classification with modern tensorflow and some fine tuning on TugsTugi's BERT model.
Load Mongolian BERT in Tensorflow 2
Generate text using Mongolian BERT
Visualize Mongolian BERT using bertviz and pytorch model
Fine tuning TugsTugi's Mongolian BERT model
On TPU mode, loading checkpoints from the file system doesn't supported by the bert and bucket should be used.
Fine tuning mongolian BERT on TPU, You need own bucket in order to finetune on TPU
Fine tune a mongolian BERT on GPU, a lot of computation needed, a low batch size matters due to memory limit
Classifiers using simple neural networks
No 02, Pretrained Word2Vec initialization from Facebook's fasttext, kind of transfer learningish. Embedding layer is not trainable in this case and with trainable embedding layer
No 03, 1D Convolution and multiple 1D convnets
Visualize RNN neuron firing in text generation
No 05, LSTM with Attention, visualization of attention scores in text classification
No 06, Classification with Mongolian BERT and Tensorflow 2.0, with frozen bert layers
No 07, Classification with Mongolian BERT large and HuggingFace and Tensorflow 2
Mongolian sentence interpolation experiments
Sequence loss in keras and tf2
Variational Auto Encoder for Mongolian text
Other experiments
Predict next word, greedy text generation
Series included(or will) followings
word2vec initialization, 1D Convolution, RNN variants, Attention, Some weights visualization for reasoning, Transformer, Techniques to handle longer texts and so on...
useful references and resources
- Mongolian BERT models https://github.com/tugstugi/mongolian-bert
- Mongolian NLP https://github.com/tugstugi/mongolian-nlp
- Eduge classification baseline using SVM https://colab.research.google.com/github/tugstugi/mongolian-nlp/blob/master/misc/Eduge_SVM.ipynb
- News crawler https://github.com/codelucas/newspaper
Images and screenshots