Tejas Vaidhya

Sorry for closing PR before Commit history of git is now updated ### News **Updated** [Demo](https://github.com/tejasvaidhyadev/ALBERT.jl/blob/master/docs/Demo.ipynb) - Contatins demo of embedding from wordpiece and sentencepiece - Demo of conversion of...

## Pretrained weights Version 2 of ALBERT converted Bson is released It doesn't contain `30k-clean.model` file (by sentencepiece) - [base](https://drive.google.com/drive/u/1/folders/1DlX_WZacsjt6O8EDaawKJ-x4RWP46Xj-) - [large](https://drive.google.com/drive/u/1/folders/1DlX_WZacsjt6O8EDaawKJ-x4RWP46Xj-) - [xlarge](https://drive.google.com/drive/u/1/folders/1DlX_WZacsjt6O8EDaawKJ-x4RWP46Xj-) - [xxlarge](https://drive.google.com/drive/u/1/folders/1DlX_WZacsjt6O8EDaawKJ-x4RWP46Xj-)

@aviks any suggestion on the roadmap mentioned above. i am also thinking of adding Tutorial folder (containing ipynb of tutorials)

added Sentencepiece unigram support

completed trainable Albert structure.

fine-tuning Training Tutorial (it's not supported GPU so far)- [here](https://github.com/tejasvaidhyadev/ALBERT.jl/blob/master/docs/TrainingTutorial.ipynb)

The above code is pretty messy and not yet refractor (for the experiment) we can drop Sentencepiece as soon as PR of ALBERT is merged Apart from that `pretrain.jl` is...

> Hi @tejasvaidhyadev can you move this PR to [TextModels](https://github.com/JuliaText/TextModels.jl) now please? Hi @aviks, Is it okay, if I will do it the coming weekend? I have exams this week

Hi @Ayushk4 CorpusLoaders.jl APIs can not be directly used because of datatype constraint input require in Avg Perceptron POS model. I think we can provide new methods in Avg Perceptron...

Hi @Ayushk4 The process is getting killed due to unknown reasons during training on the CoNLL dataset. I am attaching [gist ](https://gist.github.com/tejasvaidhyadev/01ea565ee8807a59c672866e4057c360)file of code used (Sorry, code is little messy)...