Poedator

as a temporary measure, is it safe for user to delete manually all non-relevant versions from `/site-packages/bitsandbytes` ? like this (for keeping only cuda 12.1 versions): ``` cd ~/conda/envs/py38/lib/python3.8/site-packages/bitsandbytes find...

> @poedator on the basis of the above discussion, where it looks like folks are moving the codebase in the right direction and have clarified that manual deletion is fine,...

Can confirm: when the model is loaded in safetensors format, this can reduce the memory usage by a factor of 5+. When experimenting with llama2-70b, we found that the memory...

pls see discussion by following the link in [flozi00](https://github.com/flozi00) comment.

also observed this issue on Linux, (private swarm, Bloom-560)

+1