dan_the_3rd

I believe that if you messed up something with the dimensions, there would be much bigger differences right? (like image completely dark, or random) Looking at the reference function `cross_attention_attnblock_forward`,...

Oh glad to see it working! I still don't understand how it could generate "good looking" images before while being entirely wrong lol

These reshapes will however incur an additional cost (in terms of compute), so it might be worth to evaluate xformer's speedup again.

> Tried it and got `RuntimeError: query: last dimension must be contiguous` in the call to xformers.memory_efficient_attention. Setting q, k, v to .contiguous() fixes it. This is no longer required...

> > Tried it and got `RuntimeError: query: last dimension must be contiguous` in the call to xformers.memory_efficient_attention. Setting q, k, v to .contiguous() fixes it. > > This is...

Yes, we updated [install instructions](https://github.com/facebookresearch/xformers#installing-xformers) with release of 0.0.16. Pip/conda is the recommended way to go. (note that you don't need to install `cutlass`) Closing as I think it is...

Thanks for reporting :) Should be fixed in https://github.com/facebookresearch/xformers/pull/438

Hi, the PR was merged so it should be yes. Please let us know if you have other issues

Oh this is a regression - right. The purpose of `rng_seed`, `rng_offset` is to keep the RNG state for the backward pass. This is useful when there is a dropout...

Does JIT support SymInt? Because the version in PT outputs `SymInt`, not exactly sure why. Anyway we want to rely on the PyTorch version moving forward (with the C++ code...