xformers icon indicating copy to clipboard operation
xformers copied to clipboard

[HOWTO failed] Can blocksparse attention support attention_mask?

Open SkyAndCloud opened this issue 1 year ago • 2 comments

Hi, I run the blocksparse code snippet in HOWTO.md and failed. The error message shows:

AssertionError: This attention does not support attention masks

This is caused by line. It seems that there needs to be improved for HOWTO.md. Futhermore, we may need a flexible sparse attention which supports arbitrary specefic attention mask in many cases. It would be wonderful and powerful. Can blocksparse attention support attention_mask?

SkyAndCloud avatar Mar 13 '23 02:03 SkyAndCloud

I run into the same problem. Looking forward to any possible solutions.

fuvty avatar May 28 '23 07:05 fuvty

Still the same problem

nofreewill42 avatar Feb 21 '24 19:02 nofreewill42