OmniFusion

OmniFusion copied to clipboard

Add 4bit inference code

2

Hello, I'm really interested in your project. But it takes much vram to run. Could you please add inference code in 4bits using bitsandbytes or smth

WithNoAsterisk

how to start it up ?

2

cant find any inference code. how start it working ?

zoldaten

Thanks for such a awesome work. Can you please release the detail commands for sft training and pretraining. It wAould be really helpful.

sahilqure

see above. openrouter.ai - wil be nice to have and test this model there with some standard api.

YuriyGavrilov