ROCm Backend Status Tracker

[pnunna93]

This issue summarizes and tracks the current status of ROCm enablement for BitsandBytes.

• [x] Enable ROCm backend for bitsandbytes - Supported from ROCm 6.1 https://github.com/TimDettmers/bitsandbytes/pull/1207 Updated CUDA backend to work seamlessly for ROCm. Added hipified versions of CUDA kernels and ops which allows to use optimized HIP kernels for AMD GPUs. Cmake build updated for ROCm and also enabled key features in bitsandbytes functional and autograd api
• [x] Update ROCm install instructions in official bitsandbytes documentation https://github.com/TimDettmers/bitsandbytes/pull/1244 Bitsandbytes can be built for ROCm backend by setting COMPUTE_BACKEND to 'hip' in the cmake command. The instructions to get supported dockers and steps to build ROCm enabled bitsandbytes are present in the installation doc of multi-backend-refactor branch.
• [x] Provide AMD GPU to public CI AMD GPU has been successfully added into Huggingface CI for ROCm backend testing
• [x] Enable ROCm build in public CI https://github.com/TimDettmers/bitsandbytes/pull/1255 Added a new CI job to build bitsandbytes with ROCm backend for push/pull requests to main branch, releases and manually triggered jobs
• [x] Provide packaging/wheels support for ROCm https://github.com/bitsandbytes-foundation/bitsandbytes/pull/1299 Updated shared libs and wheels build jobs for ROCm. The new updates will create ROCm specific .so files, which will be packaged into bitsandbytes wheels. In addition to this, the documentation is updated with steps to install bitsandbytes through newly built packages.
• [x] Update Debug Messages/Diagnostic Functions for ROCm https://github.com/bitsandbytes-foundation/bitsandbytes/pull/1333 The diagnostic functions which aid users with debug messages related to installation failures are updated for ROCm. Also, added additional messages for igemmlt failures and a fix to clean up runner disk space for CI jobs.
• [x] Enable packaging for ROCm 6.2 https://github.com/bitsandbytes-foundation/bitsandbytes/pull/1367 Wheels packaging is enabled for ROCm 6.2. Users can use pip install directly for ROCm 6.2 after this change.

The completed changes are in multi-backend-refactor branch and will be merged into main once multi-backend design is finalized.

cc: @Titus-von-Koeller @sunway513 @amathews-amd