picoGPT
picoGPT copied to clipboard
jaymody
An unnecessarily tiny implementation of GPT-2 in NumPy.
My VM doesn't have GPU acceleration. When I run a test command, the console has warnings like this: ` 2023-02-10 10:02:02.669259: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI...
Added Dockerfile for building a Docker image of picoGPT
Hi, @jaymody, Awesome blog post. I was interested in learning kvcache during inference and searched for it but existing articles on kvcache don't focus on the implementation part of it....
With #10, I get the following timings with NumPy on my Apple M1 Max: ```console $ time python gpt2.py "Alan Turing theorized that computers would one day become" -n 40...
Great work! For beginners, here's a graphical representation of your code. Feel free to embed it in your scripts: https://gctpy.com/graph/1ca770a1905176a355836d485ee7c8fc5b97e74ae058fce332ca59fdcf4ac919. It shows how different functions connect to other functions in...
E:\picoGPT-main>python gpt2.py "Alan Turing theorized that computers would one day become" Traceback (most recent call last): File "C:\Users\AlanT\AppData\Local\Programs\Python\Python39\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 18, in swig_import_helper fp, pathname, description = imp.find_module('_pywrap_tensorflow_internal', [dirname(__file__)]) File "C:\Users\AlanT\AppData\Local\Programs\Python\Python39\Lib\imp.py",...
Hi Jay, thank you for making the PicoGPT repo. It is very useful, as well as the blog I was following it to understand the GPT model more. The current...
A pico PR for a pico change example output: `Fetching model.ckpt.data-00000-of-00001: 21%|████▏ | 103M/498M [04:06
