mlc-llm [Question] Difference between the quantization methods of other LLM engines.

[Question] Difference between the quantization methods of other LLM engines.

Open BrandonLee0626 opened this issue 9 months ago • 0 comments

❓ General Questions

I am curious if there is a difference between the quantization methods, such as q4f16_0 and q4f32_0 of this engine, and the q4_0 quantization of other LLM engines. If there is a difference, what is it?

Jan 23 '25 10:01 BrandonLee0626

mlc-llm mlc-llm copied to clipboard

[Question] Difference between the quantization methods of other LLM engines.

❓ General Questions

mlc-llm
mlc-llm copied to clipboard