llm-awq icon indicating copy to clipboard operation
llm-awq copied to clipboard

Open-Flamingo reference

Open YerongLi opened this issue 1 year ago • 0 comments

In the paper you said the following. How to do quantization for Open-Flamingo?

Thanks to better generalization, it also achieves good quantization
performance for instruction-tuned LMs (e.g., Vicuna) and, for the first time, multi-modal LMs (Open-Flamingo [2]). Thanks to our efficient kernels, AWQ achieves 1.45× and 2× speedup over GPTQ
and GPTQ with reordering on A100.

YerongLi avatar Jul 19 '23 00:07 YerongLi