Siyuan Feng comments

Results 99 comments of


                                            Siyuan Feng

Several issues were generated in building the source code with LLVM 18

Thanks for reporting this. Could you please send a PR to fix it?

[RFC] Introduce TPAT (TVM Plugin Autogen Tool)

Can you please review this? @buptqq

[Tracking Issue] [Experimental] Relax Fallback Mechanism for Missing Operators

I'm suffering from the same issue with Bert models.

Support Qwen2-MoE Architecture

depends on https://github.com/apache/tvm/pull/16848, waiting for the next sync

Support Qwen2-MoE Architecture

Waiting for dependencies: https://github.com/apache/tvm/pull/16887 and https://github.com/apache/tvm/pull/16886 BTW, there is a known numerical issue on Vulkan. Will fix it in a follow-up PR.

[Question] Can PagedKVCache support different size of kvcache in different layers?

The current version does not support it yet and it might be hard to modify for it.

[Feature Request] run the LLM model on the Qualcomm Hexagon NPU in Android OS

I tried a bit but failed since Hexagon is not OPEN for developers. To be specific: 1. 32-bit RTOS with 4GB memory limitation. (Qualcomm can use tricks to support more...

[Feature Request] run the LLM model on the Qualcomm Hexagon NPU in Android OS

@ningpengtao-coder Thanks for your suggestion. That's a good approach to running models on Android. However, I (as well as the team) do not have extra bandwidth to support NNAPI in...

[Bug] Unexpected Error: The model weight size may be larger than GPU memory size

cc @MasterJH5574 to see if we can enhance the error message

[Model Request] Microsoft Phi-3 mini Instruct (Faster and better then LLama 3 8B)

There is a WIP PR: https://github.com/mlc-ai/mlc-llm/pull/2222