minGPT
minGPT copied to clipboard
karpathy
A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
In the case of train_dataset[0], wherein self.permutation_array[0] contains 4717, why does __get_item__ return x, y as (tensor([4, 7, 1, 7, 0, 6]), tensor([-100, -100, -100, 0, 6, 4])) and not...
1. No mpi imports, just native torch multiprocessing. 2. Adding play_char.py to test the trainer.
Hi, I am currently having proken pipe error while trying to train the model for images. While I am trying to solve this error on my machine, I would like...
Hey folks, Thanks a lot for this implementation @karpathy! I was wondering how you got the values in the addition example: ``` warmup_tokens=1024, final_tokens=50 * len(train_dataset) * (ndigit + 1),...
When running Input 12 in the play_char jupyter notebook, I got the error: --------------------------------------------------------------------------- BrokenPipeError Traceback (most recent call last) in 6 num_workers=4) 7 trainer = Trainer(model, train_dataset, None, tconf)...
@karpathy and other contributors Hey guys, I am loving your implementation. It's awesome, but I am getting an error when I try to use a test dataset during training with...
Hi, I am currently testing the char notebook. Everything works fine while CPU training, but if I try to execute the same code on a GPU/TPU the following error occurs:...
I've been experimenting with minGPT / play_math for the purpose of seeing if multiplication is possible. I've got a somewhat anemic GTX1060 with only 6GB of memory, when attempting to...
