nanoGPT icon indicating copy to clipboard operation
nanoGPT copied to clipboard

Published 20 hours ago • verified publisher icon karpathy

Hardware requirements for inference?

Open jjtolton opened this issue 2 years ago • 1 comments

I see the documentation on the hardware requirements for training. Any thoughts on what the requirements for inference are? Thank you!

jjtolton avatar Jan 11 '23 16:01 jjtolton

According to this,

the logic for the 1.5 billion param GPT-2: $1.5 \times 10^9 \textrm{params}\times 2 \textrm{bytes/param} \times 10^{-9} \textrm{gigs/byte} = 3 \textrm{gigs}$.

Does that seem accurate?

jjtolton avatar Jan 11 '23 17:01 jjtolton