ChatLaw icon indicating copy to clipboard operation
ChatLaw copied to clipboard

peft model inference so slow!!

Open KLGR123 opened this issue 1 year ago • 0 comments

image As shown, I tried set `load_in_8bit=False` or set `model = model.merge_and_unload()`, but neither work. I mean it can output result like in 2000 years later SO is there a solution yet??

KLGR123 avatar Sep 21 '23 06:09 KLGR123