Ruihang Lai
Ruihang Lai
@Moxoo Thank you for reporting! For the issue of “missing `}`”, are you running with the JSON `response_format`? Or you run with the normal text response format (if you didn't...
@Moxoo Thanks for getting back. Definitely let us know if you see further issues.
Hi @mhollis1980, thanks for reporting. we are not able to reproduce this failure so have no immediate ideas. It's also interesting to me why the linker couldn't find the .so...
Hi @afsara-ben thank you for reporting this. I just tried to build the app from fresh with the Llama 3.2 3B model, and the app works well. The specific `mlc-package-config.json`...
Hi @BlindDeveloper, we don't use nnapi. That may be something we can do in the long term roadmap, but for now we do not leverage that.
May I ask how did you install mlc? We don't have a namespace called `mlc`.
Could you folks try run with environment variable `MLC_JIT_POLICY=REDO` to force the recompilation of models? For example, `MLC_JIT_POLICY=REDO python -m mlc_llm package`. The submodule update won't automatically trigger recompilation unfortunately,...
Hi @wwt02 thank you for asking. May I ask for the content of `build/config.cmake`? Also, I noted that you're using CUDA 11.5, which is no longer supported by MLC. We...
Hi @wwt02 thank you. Indeed we will need CUDA version at least 12.
Please check related issues https://github.com/mlc-ai/mlc-llm/issues/2882, https://github.com/mlc-ai/mlc-llm/issues/2884.