gpt-fast icon indicating copy to clipboard operation
gpt-fast copied to clipboard

Published 20 hours ago • verified publisher icon pytorch-labs

Does `gpt-fast` work on V100 GPUs?

Open RomanKoshkin opened this issue 1 year ago • 2 comments

Everything works on my A6000s and A100s, but not on the older V100 (says compute capability is low). Are there plans to add support for the legacy devices? Thanks!

RomanKoshkin avatar Jan 03 '24 07:01 RomanKoshkin

We haven't tested on V100s so I'm not sure. I thought it worked but haven't checked.

Chillee avatar Feb 21 '24 21:02 Chillee

I actually tried it just now. The issue is that V100 has poor bfloat16 support. If you just change all the bfloat16 instances to float16 it should work.

Chillee avatar Feb 28 '24 23:02 Chillee