Phi-3CookBook icon indicating copy to clipboard operation
Phi-3CookBook copied to clipboard

typo about int4

Open rockets-cn opened this issue 9 months ago • 0 comments

"Using Intel NPU acceleration, this library does not affect the traditional encoding process. You only need to use this library to quantize the original Phi-3 model, such as FP16, INT4,such as" I have tried int4 but not working so I check the doc from intel doc"https://github.com/intel/intel-npu-acceleration-library". it seems that the low is int8 not int4 so maybe you have to modified to int8 like " Using Intel NPU acceleration, this library does not affect the traditional encoding process. You only need to use this library to quantize the original Phi-3 model, such as FP16, INT8,such as"

rockets-cn avatar May 25 '24 10:05 rockets-cn