mlc-llm

mlc-llm copied to clipboard

Speed benchmark compare with llama.cpp

1

hello ,does there any speed throughout benchmark comparing with llama.cpp?

luohao123

An alternative python interface for MLC LLM

3

It seems there is no official support for interacting with MLC LLM using python api yet. For the convenience of anyone who wants to develop a program that integrates MLC...

XinyuSun

Hi, do you happen to know how the optimization performance for this TVM based solution might compare to https://github.com/hidet-org/hidet for NVIDIA edge devices (like NVIDIA Jetson Xavier)? I'm curious to...

austinmw

Does it support iPhone 13

3

Does MLC support the non pro iPhone 13, or do I need to buy more RAM?

upintheairsheep

Hey, nice launch! Since LLama and its variant vicuna has commercial restriction and huggingface released openllama https://huggingface.co/openlm-research/open_llama_7b_preview_200bt. Can you support it?

salaki

Seriously what was that?

1


Alimazraeh

tvm::runtime::InternalError relax/src/runtime/relax_vm/lm_support.cc:247 Check failed: uniform_sample <= data[0].first (0.0715982 vs. nan)

5

trying to build ios app from the source and everything is ok except for running the app on the iPhone, the app shows ready to chat but after sending the...

birham-red-bd

Fixed a small typo. The two previous commits were just updating my forked branch.

Fubge

CMake Error

2

I am new to TVM, and I encountered an error while compiling it according to the [instructions](https://github.com/mlc-ai/mlc-llm/blob/main/ios/README.md). I cannot install it successfully. It seems that TVM or something else cannot...

zp2459

python chat.py can not be run

6

This actually do not provide a loadable in huggingface repo, how does the tokenizer can load? ``` OSError: ./dist/models/vicuna-v1-7b does not appear to have a file named config.json. Checkout 'https://huggingface.co/./dist/models/vicuna-v1-7b/None'...

lucasjinreal