xformers

xformers copied to clipboard

Customization of BlockDiagonalMask or BlockDiagonalCausalMask

2

# ❓ Questions and Help I am trying to write a customization of BlockDiagonalMask with an additional variable `group_lengths_list`. Inside each block, tokens are separated into 3 groups (number of...

KJ-rc

# 🐛 Bug ## Command I encounter the following error during training from time to time: ``` File "/home/qiyan/anaconda3/envs/mtr/lib/python3.8/site-packages/torch/_tensor.py", line 522, in backward torch.autograd.backward( File "/home/qiyan/anaconda3/envs/mtr/lib/python3.8/site-packages/torch/autograd/__init__.py", line 266, in backward...

qiyan98

Significant performance drop in training

4

# ❓ Questions and Help I'm new to xformers. I need to use Transformer Encoders to train on a dataset with a very large variation in sample lengths. My original...

Fan-Yixuan

Dependency Conflict with Torch and Torchvision Versions

3

When installing `xformers`, it upgrades `torch` to version `2.0.1`, even if a previous version (`1.12.1`) is already installed. This causes a compatibility issue with `torchvision 0.13.1`, which requires `torch 1.12.1`....

sakhrNab

# ❓ Questions and Help Hi, I am trying to make an encoding with `xFormerEncoderBlock` of a data sequence. The problem is that the nature of my data can have...

EloyAnguiano

[HOWTO failed] Can blocksparse attention support attention_mask?

2

Hi, I run the blocksparse code snippet in HOWTO.md and failed. The error message shows: ``` AssertionError: This attention does not support attention masks ``` This is caused by [line](https://github.com/facebookresearch/xformers/blob/b6be33aecb5297f3f994568cf29e194a75e47667/xformers/components/attention/blocksparse.py#L105)....

SkyAndCloud

2.0.1 version torch unspport

lucasjinreal

importing torch in setup.py will cause arcane exceptions when xformers is added without versions

12

https://github.com/facebookresearch/xformers/blame/0ec6aa244f3fbf1a077782bbf3ac8e6f2e4d1d16/setup.py#L23C5-L23C5 setup logs: ``` INFO: pip is looking at multiple versions of xformers to determine which version is compatible with other requirements. This could take a while. Collecting xformers (from...

doctorpangloss

fatal error: cuda_fp8.h: No such file or directory

3

# ❓ Questions and Help Hi, I was trying to build xformer from the source. But encountered an error and have no idea about it. System information: ``` No LSB...

hypily123

ScaledDotProduct cannot run on cuda:1

5

# 🐛 Bug When I run on cuda:0, everything works fine. But when I run cuda:1, the following error occurs: ``` Triton softmax kernel register spillover or invalid image caught.Deactivating...

1049451037