xformers
xformers copied to clipboard
[HOWTO failed] Can blocksparse attention support attention_mask?
Hi, I run the blocksparse code snippet in HOWTO.md and failed. The error message shows:
AssertionError: This attention does not support attention masks
This is caused by line. It seems that there needs to be improved for HOWTO.md. Futhermore, we may need a flexible sparse attention which supports arbitrary specefic attention mask in many cases. It would be wonderful and powerful. Can blocksparse attention support attention_mask?
I run into the same problem. Looking forward to any possible solutions.
Still the same problem