haitham-boxmind

Multi-GPU/CPU offloading is still not working as intended

1

I believe this issue should have been solved in #668, however it is still happening when I run "examples/quantize.py" on a finetuned version of `meta-llama/Llama-3.3-70B-Instruct`. My system has a 125GB...