moondream icon indicating copy to clipboard operation
moondream copied to clipboard
verified publisher icon vikhyat

very slow on cpu

Open ILG2021 opened this issue 1 year ago • 1 comments

I have tried four models on desktop cpu Intel(R) Core(TM) i7-10700 CPU moondream-0_5b-int4.mf.gz moondream-0_5b-int8.mf.gz moondream-2b-int4.mf.gz moondream-2b-int8.mf.gz it will cost much time to inference, at least 7s, int4 is faster than int8, can you improve the speed?

ILG2021 avatar Dec 21 '24 04:12 ILG2021

@vikhyat

ILG2021 avatar Dec 21 '24 23:12 ILG2021