dan_the_3rd

Hum that's not how it works. It's not an additive bias that gets added after the entire attention matrix is calculated. This specific bias type is recognized by the kernel,...

> a general implementation of pytorch What do you mean?

Hi @Shaistrong Windows has a limitation on the maximum path length - that's quite annoying, but it's not related to xFormers. I'm not sure how to work around this... Can't...

Hi, That's a valid point - fused sequence parallel has a higher CPU cost than non-fused operator, and this can impact performance. In general, in xFormers we don't won't to...

This is coming with this [release](https://github.com/facebookresearch/xformers/releases/tag/v0.0.26) - the wheels/conda builds should be available in a few hours :) Thanks for the heads-up!

Right good point. Doing it again now - hopefully this can be out by end of day

Hey @xiahongyu939 how did you install xformers and pytorch? There is something wrong in your setup... @JanineCHEN Can you try with xformers with cuda 12.1? `pip install -U --pre xformers`

Actually you should be able to install the cu118 version directly with: ``` pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu118 ```

Hi, Thanks for opening this post! At this point, windows support is best effort (it's not something we need internally). If you can make things compatible with windows, we would...

Hi, It looks like cuda (or at least NVCC) is not correctly installed.