Matthew Douglas

Hi @ClaireCJS, Improving this log output and messaging is something we're going to continue to work on in the future. You're right in that this is overly verbose. As for...

Closing as the logging will be tracked in #919.

We noticed that there has been no recent activity on this issue. As a result, we will be closing it for now. If you continue to experience this problem or...

Hi @mm04926412, we discussed briefly on the CUDA MODE Discord. Happy to look at any PRs for this, and ideally some data to show the impact would be appreciated! Feel...

@mm04926412 No problem! Your suggested form is more or less what I was expecting to see (except in my head I was using `__clz` instead of `__ffs`). I could take...

Closing as these changes were included in #1401 and the v0.45.0 release.

We do have a more optimal GEMV path for inference with batch size of 1, but otherwise your thought process here is sound. It should be possible, and I would...

@sidhantls Please see the answer here: https://github.com/bitsandbytes-foundation/bitsandbytes/issues/1400#issuecomment-2434081536 In short, outliers in the activations are kept in fp16. The corresponding channels in the weights are dequantized (with some error) from int8...

Related: pytorch/pytorch#124245 I'm not sure what PyTorch release that's going to land in, but good news is that it looks like it'll eventually support MSVC on Windows and we can...

@Titus-von-Koeller That's just a correction on comment about the compiler requirement for Windows (when we get there). It will use MSVC by default and not clang. I think it's OK...