YLSnowy

Can we make it run on other models?

6

Can we make it run on other models? Can the offload code be made public?

How to split the model parameter safetensors file into multiple small files

5

The original Mixtral model safetensors files have a total of 19, and the parameter file you provided has 257 safetensors. How do you split the model? Can you provide this...

run on the mutiple gpus

3

Is there an unquantized version that can run on multiple GPUs?