Stephen888888

Results 8 comments of Stephen888888

[Bug] The performance accuracy of large models is severely lost after quantization on Qwen2-1.5B-Instruct ，please fix it

Model is complie as below: mlc_llm convert_weight ./dist/models/Qwen2-1.5B/ --model-type qwen2 --quantization q4f16_1 -o dist/Qwen2-1.5B-q4f16_1-MLC mlc_llm gen_config ./dist/models/Qwen2-1.5B/ --model-type qwen2 --quantization q4f16_1 --conv-template chatml --context-window-size 2048 --max-batch-size 1 -o dist/Qwen2-1.5B-q4f16_1-MLC/ my...

[Bug] The performance accuracy of large models is severely lost after quantization on Qwen2-1.5B-Instruct ，please fix it

log shows as below: 2024-06-12 16:43:11.778 10890-10890 HwRemoteIn...hodManager ai.mlc.mlcchat W isCasting false because IHwDistributedWindowManager is invalid. 2024-06-12 16:43:11.788 10890-10890 DecorView ai.mlc.mlcchat I navBarColor: fffcfcfc statusBarColor: ff00668b statusInsets: Insets{left=0, top=91, right=0,...

[Bug] The performance accuracy of large models is severely lost after quantization on Qwen2-1.5B-Instruct ，please fix it

@jeethu any problem found ？

[Bug] The performance accuracy of large models is severely lost after quantization on Qwen2-1.5B-Instruct ，please fix it

any problem found ？

[Bug] The performance accuracy of large models is severely lost after quantization on Qwen2-1.5B-Instruct ，please fix it

any problem found ？

Source for v2 (mobile inference engine)

when can i use it on anroid phone ?

[Bug] Does MLC LLM support function call ？

@panghongtao yes，already run on android phone. but i need function call

[Bug] Does MLC LLM support function call ？

@panghongtao yes，can you give me a demo interface ？