Junru Shao comments

Results 179 comments of


                                            Junru Shao

no enough to run app for iPhone 14 pro max

This is weird because my 14 pro runs properly without this issue. 14 ProMax comes with 6GB memory, which means they should work smoothly without the issue you guys mentioned.

no enough to run app for iPhone 14 pro max

This should work now if you upgrade to the latest iOS :-)

could you tell me to support Chinese dialogue?

Hey thanks for asking! MLC LLM doesn't limit which language to use per se, but good language support really depends on what underlying model it is generating code for. For...

"As an AI language model,"...

MLC LLM is a compiler, which doesn't control the text generation from models per se. Currently the "as an AI language model" is directly generated from Vicuna-7b and we didn't...

Does mlc-llm support nv gpu using CUDA instead Vulkan

We haven't particularly focused a lot on CUDA optimization yet, as `torch.compile()` should already work for huggingface models out of the box. Happy to bring in more optimization later

how to build mlc-llm-cli on Linux

Please check out this page for building `mlc_chat_cli`: https://mlc.ai/mlc-llm/docs/tutorials/runtime/cpp.html

PackagesNotFoundError: mlc-chat-nightly

Please check out this page for building `mlc_chat_cli` properly from source: https://mlc.ai/mlc-llm/docs/tutorials/runtime/cpp.html

How to build the "cpp" dir for android ?

Please refer to this documentation for details: https://mlc.ai/mlc-llm/docs/tutorials/runtime/android.html

which commit of relax should be used ?

Please check out this page for proper installation of TVM: https://github.com/mlc-ai/mlc-llm/blob/main/docs/install/tvm.rst

Implement Python chat module and REST API

very cool work!