Tim Dettmers

Looking again at this error, I realize the problem is likely that you set the memory threshold too low in the max_memory. You are currently using 3 GB per GPU,...

I believe the main issue here is that you need to use the `max_memory` dictionary as an argument. By default, it can be that the dictionary allocates too much memory...

it is as expected that threshold 0 and 6 use the close to same memory with the current implementation. The difference should be in the order of a couple of...

Currently, the library does not offer Windows support. It would be great if you can help us to make it work under Windows. For that, the first step is to...

@tarushbansal there are two issues: 1. Your GPU has compute capability of 6.0 which is currently does not support int8 matrix multiplication (you need a newer GPU). 8-bit optimizer are...

You can read more about these quantization methods in the [8-bit optimizer paper](https://arxiv.org/abs/2110.02861). You can find quantile quantization in the appendix. None of the block-wise quantization methods preserve distances or...

I personally do not have time to implement ROCm support. If you have experience with ROCm I could guide you through the steps to get a working solution.

Thanks for the questions! The model that we trained were autoregressive language models, only the corpus that we use was from RoBERTa. The baselines models come from the [BASE layer...

It seems a specific compiled version has not been found even though it should be available. Can you please provide the output for the command ```bash ls /mnt/dolphinfs/hdd_pool/docker/user/hadoop-speech/users/lisong39/python_packs/lib/python3.8/site-packages/bitsandbytes-0.32.1-py3.8.egg/bitsandbytes/ ```

I believe this is fixed in the latest version. It prints instructions on how to debug the situation and alternatively prints out compilation instructions which should fix the issue.