electron271
electron271
i have found a likely solution, if it works maybe i can switch over the builds to my fork until its merged in so 4bit works
marking as draft until i get this issue fixed as it is fairly major
pr created: https://github.com/bitsandbytes-foundation/bitsandbytes/pull/1748
should work now, testing changes
works
> Works great on AMD MI100. > > I added this to my vllm Dockerfile and it just worked. > ```yml > RUN git clone --recurse https://github.com/ROCm/bitsandbytes && cd bitsandbytes...
> I ran `git clone https://github.com/electron271/unsloth-rocm.git && cd unsloth-rocm && pip install .[rocm-torch280]` on MI300x and tried to do DPO and get this error: > > ``` > Traceback (most...
> I like the way to provide our end user the fresh prebuilt bnb binary directly in the patch. Somehow this does not work in some environment That's one of...
the upstream bitsandbytes pr should hopefully be able to be merged soon
> Hi @electron271 You'll want to try to build on Ubuntu 22.04 instead of Ubuntu 24.04 to have better compatibility - your repo is producing wheels with a glibc 2.39...