[Guide] How to finish xFormers compile with source code, update 3rd-party compoents, or Nvidia Docker envs

[soulteary]

I have seen many users in the community encounter compilation and build problems, and many failures, especially when using new versions of CUDA, PyTorch, or wanting to update CUTLASS or Flash Attention.

So I wrote a tutorial on how to use Docker (Nvidia's monthly release, including the latest CUDA, Torch, etc.) to fully install and build xFormers.

guide

• https://soulteary.com/2024/01/12/xformers-source-code-compilation-with-nvidia-docker.html
• (backup) https://zhuanlan.zhihu.com/p/677516241
• dockerfile example: https://github.com/soulteary/docker-stable-diffusion-webui/blob/main/docker/Dockerfile.xformers

I believe that people who have encountered the same problem should be able to complete the construction by drawing on the ideas in the article.

If you like reading in English, I recommend using Google Translate to translate the article.

However, if you don’t translate it, but refer to the command line in the article, it should be enough. I wish everyone good luck and hope it can save time.

main steps

download complete source code, includes xformers, flash attention, cutlass...

git clone --recursive https://github.com/facebookresearch/xformers.git --depth 1

update 3rd-party source code.

# we can update to latest
cd xformers/third_party/flash-attention
git pull origin main

# update to flash attention match version (optional)
# cd xformers/third_party/cutlass
# git pull origin main
# git checkout v3.3.0

set git config

git config --global --add safe.directory /app/xformers
git config --global --add safe.directory /app/xformers/third_party/flash-attention
git config --global --add safe.directory /app/xformers/third_party/cutlass

install build deps:

pip install ninja

enjoy

pip install -v -e .

final, python -m torch.utils.collect_env

xFormers 0.0.24+6600003.d20240112
memory_efficient_attention.cutlassF:               available
memory_efficient_attention.cutlassB:               available
memory_efficient_attention.decoderF:               available
[email protected]:        available
[email protected]:        available
memory_efficient_attention.smallkF:                available
memory_efficient_attention.smallkB:                available
memory_efficient_attention.tritonflashattF:        unavailable
memory_efficient_attention.tritonflashattB:        unavailable
memory_efficient_attention.triton_splitKF:         available
indexing.scaled_index_addF:                        available
indexing.scaled_index_addB:                        available
indexing.index_select:                             available
swiglu.dual_gemm_silu:                             available
swiglu.gemm_fused_operand_sum:                     available
swiglu.fused.p.cpp:                                available
is_triton_available:                               True
pytorch.version:                                   2.2.0a0+81ea7a4
pytorch.cuda:                                      available
gpu.compute_capability:                            8.9
gpu.name:                                          NVIDIA GeForce RTX 4090
dcgm_profiler:                                     unavailable
build.info:                                        available
build.cuda_version:                                1230
build.python_version:                              3.10.12
build.torch_version:                               2.2.0a0+81ea7a4
build.env.TORCH_CUDA_ARCH_LIST:                    5.2 6.0 6.1 7.0 7.2 7.5 8.0 8.6 8.7 9.0+PTX
build.env.XFORMERS_BUILD_TYPE:                     None
build.env.XFORMERS_ENABLE_DEBUG_ASSERTIONS:        None
build.env.NVCC_FLAGS:                              None
build.env.XFORMERS_PACKAGE_FROM:                   None
build.nvcc_version:                                12.3.107
source.privacy:                                    open source

@danthe3rd , hope it helps, and maybe you can fixed this topic, i think there're many people failed on build from source.