Garrett Byrd

@rocking5566 Not everyone wants to use Docker all the time. Also, does the ROCm installation work with ninja to do multithreaded compilation? I have seen others posting about long compilation...

@rocking5566 Right now for AMD GPU/ROCm Support all that is listed under requirements is: > Requirements: > - ROCm 6.0+ > - PyTorch 1.12.1+ This is just a partial list...

The main install instructions do mention `packaging` and `ninja` ([link](https://github.com/Dao-AILab/flash-attention?tab=readme-ov-file#installation-and-features)): > 1. Make sure that PyTorch is installed. > 2. Make sure that `packaging` is installed (`pip install packaging`) >...

@rocking5566 Actually an even more elegant solution would just be to provide different subsections for CUDA and ROCm support in the `Installation and Features` section. (Currently there are two `Installation...

Went ahead and made a [PR](https://github.com/Dao-AILab/flash-attention/pull/1147).

PR was merged, so I'm closing this.

I have traced the breaking point as follows: From the original script, `repeats` is eventually passed to `flash_attn.utils.benchmark.benchmark_fwd_bwd` in the line `m = t.timeit(repeats)`. The `benchmark_fwd_bwd` function calls `benchmark_backward` from...

`bitsandbytes` seems to install correctly, but I have encountered an issue when testing. I am following [this blog](https://rocm.blogs.amd.com/artificial-intelligence/llama2-lora/README.html). In [this block](https://rocm.blogs.amd.com/artificial-intelligence/llama2-lora/README.html#training-with-lora-configuration) ``` from peft import get_peft_model # LoRA Config peft_parameters...

@SeanSong-amd I will test with the up-to-date branch as discussed [here](https://github.com/ROCm/ROCm/issues/3447) and then provide a description of any versioning issues.

I can confirm this works using the [`multi-backend-refactor` branch of `bitsandbytes`](https://github.com/bitsandbytes-foundation/bitsandbytes/tree/multi-backend-refactor). Thanks for the assistance. For posterity, I will list my versions below. ``` $ pip freeze absl-py==2.1.0 accelerate==0.33.0 aiohappyeyeballs==2.3.4...