joeshmoe0112358

Adding WikiText-103 dataset preprocessing and tokenization

9

This PR enables the preprocessing and tokenization of the WikiText-103 dataset in response to #246. A few notes I would like to make on my preprocessing of the text: -...

Adding GPT2 model evaluation on WikiText-103 with optional preprocessing in dev/model-eval/

2

**Ultimately in response to #246:** In regards to this specific PR, I propose the addition of two files in a new ```dev/model_eval/``` folder. One file prepares wikitext-103 for a model's...