nanoGPT
nanoGPT copied to clipboard
karpathy
The simplest, fastest repository for training/finetuning medium-sized GPTs.
This is a question that has been bothering me for a while and so I decided to ask here (hope that's fine n_n). The masking scheme that is used here...
If `decay_lr = True` and `iter_num = 0` (i.e. an initialization from scratch), the effective `lr` for the first iteration is 0 because of `it / warmup_iters` in `get_lr`. This...
When i try to run sample.py with shakespeare sample data i get: Traceback (most recent call last): File "C:\Users\joachim.bischof\nanoGPT\nanoGPT\sample.py", line 23, in exec(open('configurator.py').read()) # overrides from command line or config...
below is the error i am getting. no matter what content i put in the start as the promt. i always get this error. even with the example model out-shakespeare-char...
nanoGPT# python3 data/shakespeare_char/prepare.py /usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (2.2.1) or chardet (3.0.4) doesn't match a supported version! warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported " length of dataset in characters:...
How can I train nanoGPT using TPU's? Can I just modify the DDP targeting TPU VM's or need to make changes to the model to make it XLA compilable?
Hi There. I am looping though the model generate within sample.py to create a temperature profile of the model. I am looping through a range of min and max temperatures....
First, thank you for creating nanoGPT. It has been an amazing learning experience! I have a question about vocab size and training. I have built nanoGPT and ran the Shakespeare...
I am working on using NanoGPT to solve a geometry problem. I would like to use the gpt2 network structure but my own tokenizer. My vocabulary size is 1500. I...
