Siyuan Feng comments

Results 99 comments of


                                            Siyuan Feng

[Target] Support CUDA device function calls

Thanks @tqchen and @Kathryn-cat 's valuable comments, will refactor the PR to enhance SplitHostDevice systematically.

[Target] Support CUDA device function calls

@tqchen @Kathryn-cat I've updated a version to detect if a function is from hthe ost side and device side at an early stage (in BindTarget pass), here is an example...

[FFI] Rebase tvm to v0.22.0 to utilize tvm-ffi

Thanks @LeiWang1999

[TIR] Enhance `LowerThreadAllreduce` pass to automatically infer shared memory scope

Please add a testcase for the enhancement

gpu Memory utilisation Error, i have seen same error in 12GB & 8GB RAM device for llama3.1 and gemma 9b quantization q4f16_1

Not sure but some android vendor OS only provides limited memory for GPU but not all of the DRAM

[Bug] chatglm4 mlc_llm shows error "TVMError: Check failed: append_length > 0 (0 vs. 0) : Append with length 0 is not allowed." during mlc_llm chat CLI

I'm not sure but GLM may use a customized tokenizer which is not supported yet

[Bug] The quick start example fails because tvm.compile is missing

Thanks for pointing it out. The docs is based on the latest version (aka main branch) but not the specific version. While we are finding out why the docs for...

[Bug] The Qwen2-1.5B-Instruct cannot run on Android phones with Huawei Mate50 (8gen1)

Could you please try to run on different devices? Not sure but I heard that Huawei's OpenCL runtime is different from other android devices. And we did not test the...

[Question] How to optimize the scheduling of multimodal LLM model convolution in mlc.

It would be great if you can provide the copyable kernel script :)

[Relax] Fix the parser to avoid treating a list as an integer

Thanks for the contribution! First, it's not a typical **bug**. `(10)` is just an integer rather than a list based on the Python syntax, while `[10]` and `(10, )` are...