Sequoia icon indicating copy to clipboard operation
Sequoia copied to clipboard

Published 20 hours ago • verified publisher icon Infini-AI-Lab

Work On CPU

Open ZepinLi opened this issue 1 year ago • 1 comments

Hi Sequoia team,

Can this code framework fit in cpu devices? If so, how can we do it? Any insights?

Regards

ZepinLi avatar May 21 '24 08:05 ZepinLi

I do not think this can be used on CPU. CPU does not have such high FLOPS for speculative decoding, so even if the code can run on CPU, no speed up will be found, unless you talk about some very advanced CPUs (I have no ideas about them).

dreaming-panda avatar Oct 27 '24 03:10 dreaming-panda