dan_the_3rd

Hi, This is not something supported efficiently by xFormers at the moment. You will need to create a custom kernel to do that. It might be similar to what was...

We have no plans to support it for now

You can have a look at the installation instructions: https://github.com/facebookresearch/xformers/tree/main#installing-xformers If you build from source, you might want to add the `-v` option to pip install to see the build...

Hi, The most recent versions of xFormers only support PyTorch 2.1+. You can build older versions from source if you have PyTorch 1.13 for instance, but I would recommend that...

Hi, This is a fairly old code that we won't improve moving forward (and it will most likely become [deprecated](https://github.com/facebookresearch/xformers/issues/848)). I would encourage you to try using torch compile for...

Hi, You need to install PyTorch first, before you can install xFormers.

I'm not sure about this error - maybe @fmassa you know who would be the right POC there? Also, do you have a stacktrace for the error @1049451037 ?

Hi @asdfd2013 I don't see any bug from looking at your code, I expect `attn_weights_v1` and `attn_weights_v2` to be very close. However they won't be identical, due to how numerical...

Thanks for reporting! Do you mind sharing what is the resolution you are using? Also can you report the output of `python -m xformers.info` ?

Thanks a lot - this is really useful! @artkorenev has been working on this and should have a fix coming soon