xformers

xformers copied to clipboard

Poor performance of sequence parallel fused kernels in real model training

4

# ❓ Questions and Help I just found fused kernels in sequence parallel got poor performance in real model training. Here is a snapshot of nsys timeline of a ColumnParallelLinear...

Fanoid

# 🚀 Feature A MLP implementation that efficiently avoid computing for masked tokens. ## Motivation The motivation is that if padding is quite unequal too much MLP computation is wasted...

nicolas-dufour

# 🚀 Feature When `seqlen` is not fixed, `triton.autotune` is triggered for every unseen value of `seqlen` for tiled matmul kernel and matmul kernel. It makes the training extremely slow...

Fanoid

Reproduce attention result by pytorch failed

2

# ❓ Questions and Help I try to use torch to reproduce the result of memory_efficient_attention by following [api docs](https://facebookresearch.github.io/xformers/components/ops.html).But didn't successful, It maybe cause from my negligence or bug？...

asdfd2013

Extremely slow while building xformers from source

10

# ❓ Questions and Help Building **xformers** from source extremely slows down my computer ... freezing almost everything. How much memory does **xformers** need to be built from source?

jiapei100

Xformers are slowing down on 3090/4090

1

# ❓ Questions and Help run https://github.com/facebookresearch/xformers/blob/main/xformers/benchmarks/benchmark_transformer.py but more slower. my env: pytorch 2.2.1,cuda 12.1.1   

rikeLiu

importing `xformers.ops` implicitly initializes CUDA context

3

Currently, importing `xformers.ops` will implicitly initializes CUDA context. This has an unpleasant effect that we cannot use the "fork" multi-processing method. The line of code that initializes CUDA context is...

function2-llx

# 🐛 Bug ## Command ## To Reproduce Steps to reproduce the behavior: 1. 2. 3. ## Expected behavior ## Environment Please copy and paste the output from the environment...

henrywoo

no model named 'torch'

38

I'm really not used to using command prompt and I'm guessing this is an issue with torch but i even reinstalled it and im still getting this error. Any ideas?...

DeForestDiamond

[ROCm/xformers] error generated when compiling for gfx1100

8

# 🐛 Bug I put up some screenshots that I think it is important:        And        ##...

Looong01