multifit
multifit copied to clipboard
n-waves
The code to reproduce results from paper "MultiFiT: Efficient Multi-lingual Language Model Fine-tuning" https://arxiv.org/abs/1909.04761
exp = multifit.from_pretrained("de_multifit_paper_version") cls_dataset = exp.arch.dataset(Path('data/de_sentiment'), exp.pretrain_lm.tokenizer) cls_dataset.load_clas_databunch(bs=exp.finetune_lm.bs).show_batch() data/de_sentiment , path has train.csv/test.csv with labels, text as columns, even by shuffling as well show batch is tokenizing, Not sure why...
Had the following errors otherwise: ``` Saving data in data Chosen language: it data/wiki_dumps/itwiki-latest-pages-articles.xml.bz2 already exists. Skipping download. data/wiki_extr/it already exists. Skipping extraction. /src/fastai/.venv/bin/python: No module named multifit.create_wikitext /src/fastai/.venv/bin/python: No...
Hello everybody, Anyone knows how to get the activations of an intermediate layer when I make a prediction, despite what I have tried so far, it gives me errors.To give...
Hi everyone, I am working on the creation of saliency maps for multifit classification models, with the ultimate goal of highlighting over the text those parts that are decisive in...
I have some other problems to run the notebook CLS-DE.ipynb. If I use conda and install the default pytorch (1.3.1), after the command `exp.finetune_lm.train_(cls_dataset, num_epochs=20)` I get following error message:...
I tried replicating results for zero-shot learning on CLS, but my results don't match those from the paper. Since the script for predicting labels with LASER seems not be a...
Hi everyone. Maybe you can help me with something. Once the multifit model is trained with my own dataset, I export it to a .pkl file, in order to use...
## What I did? * Checked out the `pretrain-lm` branch because it has clear instructions how to pretrain LM (#57). * Installed required packages. * Executed `bash prepare_wiki.sh de` *...
Add a `BiAttentionPoolingClassifier` (self attention for pooling linear classifier) as in [Attention is all you need](https://arxiv.org/abs/1706.03762) following the discussion with @sebastianruder in Teams. I ran out of memory on my...
It seems that MultiFiT prefer fine-tuning to cross-language pre-training models.
