nanoGPT icon indicating copy to clipboard operation
nanoGPT copied to clipboard

Published 20 hours ago • verified publisher icon karpathy

How fast is it compared to megatron, deepspeed and something like that?

Open 2catycm opened this issue 1 year ago • 1 comments

They say they have pipeline parallel, tensor parallel, data parallel and something like that. Can nanoGPT be faster than such parallels?

2catycm avatar Apr 27 '23 15:04 2catycm