nanoGPT icon indicating copy to clipboard operation
nanoGPT copied to clipboard

[Q] Async prefetch next batch while model is doing forward pass

Open GM-git-dotcom opened this issue 9 months ago • 1 comments

Hi,

Love the repo for the self-explanatory code and fun comments!

I would appreciate it if you could explain how the next batch is asynchronously being fetched in train.py:

https://github.com/karpathy/nanoGPT/blob/325be85d9be8c81b436728a420e85796c57dba7e/train.py#L299-L305

At the outset, without explicitly using something like asyncio, the execution seemed sequential to me. Apologies for the potential triviality, thanks!

GM-git-dotcom avatar May 23 '24 22:05 GM-git-dotcom