stable-diffusion.cpp icon indicating copy to clipboard operation
stable-diffusion.cpp copied to clipboard

Published 20 hours ago • verified publisher icon leejet

metal-flash-attention support

Open czkoko opened this issue 2 years ago • 5 comments

Can this project help for you? https://github.com/philipturner/metal-flash-attention

So far, metal-flash-attention can indeed provide the fastest generation speed for stable diffusion on MacOS.

czkoko avatar Aug 21 '23 15:08 czkoko

Thank you for the feedback. I'm currently focusing on making it run faster, and I'll make time to take a look at this project and see if I can offer any assistance.

leejet avatar Aug 21 '23 15:08 leejet

I came here to suggest the same thing.

sroussey avatar Aug 21 '23 17:08 sroussey

linking this here for reference https://github.com/ggerganov/ggml/issues/293

Green-Sky avatar Aug 21 '23 18:08 Green-Sky

What needs to be done to make this happen? I'm not very good with cpp, but I want to help.

GaidamakUA avatar Oct 03 '23 06:10 GaidamakUA

https://github.com/leejet/stable-diffusion.cpp/pull/386 I am in the dark on metal flash attention support, or metal support in general. So would be nice if someone with the hardware could test the pr. :)

Green-Sky avatar Sep 12 '24 08:09 Green-Sky