OmniFusion icon indicating copy to clipboard operation
OmniFusion copied to clipboard

Published 20 hours ago • verified publisher icon AIRI-Institute

Add 4bit inference code

Open WithNoAsterisk opened this issue 10 months ago • 2 comments

Hello, I'm really interested in your project. But it takes much vram to run. Could you please add inference code in 4bits using bitsandbytes or smth

WithNoAsterisk avatar Apr 10 '24 12:04 WithNoAsterisk