torchchat icon indicating copy to clipboard operation
torchchat copied to clipboard

Published 20 hours ago • verified publisher icon pytorch

[FEATURE REQUEST] Create sdpa_with_kv support for float16, bfloat16

Open mikekgfb opened this issue 9 months ago • 1 comments

see run here => https://github.com/pytorch/torchchat/actions/runs/8872459136/job/24356835073

We can always upcast to make things pass, but if there's an easy way to build float16 and bfloat16 flavors (iPad Pro has M-series chip with bfloat support AFAIRK?), that'd be rad.

mikekgfb avatar Apr 29 '24 16:04 mikekgfb