dont-stop-pretraining
dont-stop-pretraining copied to clipboard
allenai
Code associated with the Don't Stop Pretraining ACL 2020 paper
Hi, after downloading the dataset. I want to know is there any post processing about it ? This is the keys of each dataset. In `ag` dataset should the `text`...
Hi, I am trying to train the biomed_roberta_base model on the chemprot dataset using the provided scripts.train python command and encounter the below issues.  The above...
Could you please check the implementation steps you provided in the README file? I followed your instructions but find it very hard to reproduce this work, someerrors would come out...
After use my own corpus to do domain-adaptive pretraining, the `vocab.txt` is the same size with the initialized model(BERT-base). In short, the domain-adaptive pretraining does not extend the vocabulary of...
As the authors mentioned in README.md, pytorch-transformers 1.20 is not compatible with the specified branch in [environment.yml](https://github.com/allenai/dont-stop-pretraining/blob/master/environment.yml). I tried: - Downgrading pytorch-transformers to 1.10 - install master branch of allennlp...
Hi, When following the instructions in DATA_SELECTION.md, upon running the command ""python -m scripts.train --config training_config/vampire.jsonnet --serialization-dir model_logs/vampire-world --environment VAMPIRE --device 0 -o", I get the following error: > ImportError:...
Thanks for your excellent work. I select data following DATA_SELECTION.md. During extracting VAMPIRE embeddings, one should run 'python -m scripts.run_vampire ......'. However, the file 'scripts/run_vampire.py' does not exist in the...
Traceback (most recent call last): File "/.virtualenvs/bort_test/lib/python3.6/site-packages/allennlp/common/params.py", line 239, in pop value = self.params.pop(key) KeyError: 'data_loader' During handling of the above exception, another exception occurred: Traceback (most recent call last):...
Hi, When I was running the vampire training, I found there are some issues due to the consistent version of allennlp and vampire. If allennlp==0.9.0, it says "no import module...
We want to report issues that could affect the reproducibility of the masked LM loss calculation at test time. First, we do not get exactly the same results reported in...
