nanoGPT
nanoGPT copied to clipboard
karpathy
The simplest, fastest repository for training/finetuning medium-sized GPTs.
Hey, all! I purchased a Google Coral and wanted to start exploring AI / ML This is hardly an issue, more a question: Is Google Coral compatible with Torch? Can...
how can i create a custom openwebtext file for non-English language ? like Arabic ? I need openwebtext File format
I believe this should not change the behaviour of the code. But it should use less memory and be a bit faster since it does not need an additional array.
I created an OpenWebTextCorpus DataLoader for training, and I thought you might find it useful. It automatically downloads the tar file from google drive, which is a 12GB compressed file...
Hi, I am not an expert in Transformers or DNN, but I followed the steps for training tiny_shakespeare and sampled some tokens after training. I see that it is generating...
Who has the same issue? Downloading and preparing dataset openwebtext/plain_text to C:/Users/liux3790/Desktop/download/cache/openwebtext/plain_text/1.0.0/85b3ae7051d2d72e7c5fdf6dfb462603aaa26e9ed506202bf3a24d261c6c40a1... Downloading data: 100%|█████████████████████████████████████████████████████████████████████████████████| 12.9G/12.9G [46:33
Hi, Thanks so much for nanoGPT. I have installed the dependencies and trying out the `prepare.py` code on python 3.10.4 virtual env. I see the following errors, seems related to...
GradScaler has an argument for enabling/disabling the scaler. When disabled, ``scaler.step()`` simply invokes ``optimizer.step()``, and the other methods are no-ops. I thought this made the code a bit cleaner by...
Hi. I tried to add more information to the context than just the average of the previous tokens. I added a table that calculates the distance between tokens and adds...
