Junru Shao comments

Results 179 comments of


                                            Junru Shao

[Bug] python -m mlc_chat.rest ERROR

Seems that you didn't build any `libmlc_llm_module.so`..Could you double check? Also CC @sudeepag @Kathryn-cat if you guys could help elaborate how to obtain `libmlc_llm_modyle.so`

Failed to detect local GPU

``` free(): invalid pointer 중지됨 (코어 덤프됨)

[Bug] make[2]: *** [tokenizers/CMakeFiles/tokenizers_c.dir/build.make:71: tokenizers/release/libtokenizers_c.a] Error 1

Did you install rust and cargo?

[Question] No module named 'tvm'

Please refer to this doc for TVM installation: https://mlc.ai/mlc-llm/docs/install/tvm.html

[Bug] Python server example runs but hangs on prefill function within api call

CC: @sudeepag @Kathryn-cat

[Model Request] sources for RedPajama-3b and Vicuna-7B

I'm not sure if I fully capture what you mean. Let me try: Currently we release MLC-generated model libs via this repo: https://github.com/mlc-ai/binary-mlc-llm-libs, but as you just mentioned, we haven't...

[Bug] Run RedPajama model failed on MacOS

@ghostsun89 does it work with the latest update?

Cannot auto detect device-name

I cannot follow if none of the details are provided, but according to the discussion, the problem seems to have been resolved

[Question] Is it compatible with flutter?

To interact with MLC-compiled LLMs on iOS, the only file you need is LLMChat.mm: https://github.com/mlc-ai/mlc-llm/blob/main/ios/MLCChat/LLMChat.mm. Not knowing much about flutter, but as long as it allows us to call those...

[Bug] compilation for CUDA fails on Linux with CUDA 11.8

I believe this issue will be gone following Zihao's suggestion: https://github.com/mlc-ai/mlc-llm/issues/339#issuecomment-1579587863. Please feel free to create a new one if it persists.