dan_the_3rd

If this issue only appears on the build of flash attention, you can disable its build with: ``` XFORMERS_DISABLE_FLASH_ATTN=1 pip install -e -v . ``` cc @tridao are you aware...

Have you tried disabling Flash Attention as I suggested above?

The conda errors can be cryptic and unrelated to the actual problem. What is the pytorch/python version you are using? We only support pytorch 1.12.1/1.13 and python 3.8/3.9/3.10 at this...

Thanks! Unfortunately the main issue here is that we are running out of storage on anaconda, and won't have space to store extra versions :/ Hopefully we can manage to...

How do you install xFormers? Do you install it with the `--index-url https://download.pytorch.org/whl/cuXXX` part in the pip install command? If so, we don't have any space problem. We might want...

conda is usually the better solution, but they also have the lowest storage (~10GB). We were able to [negociate](https://github.com/pypi/support/issues/2907) 20GB for pypi - trying to push it further now. The...

I'm working on getting py3.12 support for pip wheels (at least on the PyTorch s3) - hopefully we can enable that soon :)

Hi, Indeed padding sequences is not efficient for multiple reasons. What you should do instead is concatenate all of your sequences together, and when you do the attention you pass...

@function2-llx can you describe with more details what your bias look like? Like with an example for instance. It's usually better if you can avoid padding, and in fact we...

So if you want to use xFormers like this, you will need to cat patches together. Eg instead of having a shape [B, max(L_i), D], something like [1, sum(L_i), D],...