xformers

xformers copied to clipboard

Batch size >= 65536 in xformers.ops.memory_efficient_attention gives CUDA error.

8

# 🐛 Bug Xformers gives a CUDA error like this when the batch size is larger or equal to 65536. ``` RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors...

comfyanonymous

# 🐛 Bug When using block diagonal attention, with bfloat16 as the datatype and when setting torch to deterministic mode, some of the computed gradients are NaN. ## Command Save...

nimia

# ❓ Questions and Help Hi, I tested memory_efficient_attention with the pytorch equivalent implementation in the doc, and found they are not exactly the same. The code: ``` def attention_e(self,...

wangh09

[Bug] benchmark shows 0.00MB consumed

2

# 🐛 Bug benchmark shows that the memory consumption of some linear/activation layers is 0.00MB ## Command `pytest benchmark/benchmark_sp24.py` ## To Reproduce Steps to reproduce the behavior: 1.`pytest benchmark/benchmark_sp24.py` ##...

deciding

efficient_attention_forward_cutlass op is incompatible with Torch JIT

2

# 🐛 Bug RuntimeError: unsupported output type: int, from operator: xformers::efficient_attention_forward_cutlass ## Command ## To Reproduce StableDiffusionXL model use with torch.jit and enable_xformers_memory_efficient_attention function ``` m.enable_xformers_memory_efficient_attention torch.jit.trace(m) ``` Steps to...

philipwan

Implementation ideas for equivalent replacement from xformers to pytorch

3

# ❓ Questions and Help Independently implementing the equivalent replacement from xformers to pytorch would be a great engineering practice for greatly improving the underlying capabilities of network design. May...

tzayuan

ERROR: Could not build wheels for xformers, which is required to install pyproject.toml-based projects

1

# ❓ Questions and Help i use anaconda in linux.torch2.1.2+cu118 i see another issue use venv to solve it, but for weird reason, i cant use venv because when i...

greasebig

# ❓ Questions and Help Hi I tried to call xformers's Swiglufusedpacked op on my codes and get the following error during training:  By searching on web with keywords...

yup3antarctica

Install xformers without changing pytorch version

2

Hello, Is there a way to install xformers without changing Pytorch? My torch version is [2.0.0a0+1767026]. And, other package, associate with training pipeline, depends on it. So it causes other...

yup3antarctica

Memory efficient way to encode varying input sequence lengths in self attention

12

Hi, I have the issue that my sequences are strongly varying in length, with sometimes having outliers that are an order of magnitude longer than the average. In default pytorch...

thoglu