Tianqi Chen comments

Results 637 comments of


                                            Tianqi Chen

[Feature Request] Change OpenAI protocol default value to NOT_GIVEN

let us also confirm if it is the case for JSONFFIEngine

[Feature Request] Change OpenAI protocol default value to NOT_GIVEN

Just to followup on the case of JSONFFIEngine. The main purpose of JSONFFIEngine is that we should avoid passing in object and parsing mlc-chat-config from FFI side. so the current...

[Feature Request] Change OpenAI protocol default value to NOT_GIVEN

@MasterJH5574 would be good to confirm the state of this issue now in JSONFFI

[Question] Why read generation config in every decode step?

the latest MLCEngine should support concurrent generation and read config ones, see #2217

[Question] Can PagedKVCache support different size of kvcache in different layers?

KV cache is a common interface, the solution right now would be to create a difference instance of kv cache implementation of the same interfaceand replace it

[Feature Request] run the LLM model on the Qualcomm Hexagon NPU in Android OS

This is something ideally we would like to enable, and indeed we need to overcome some of the hurdles mentions. We can keep this issue open to see the status,...

[Bug] Unexpected Error: The model weight size may be larger than GPU memory size

Thanks for reporting. As a temp measure. Reduce the prefill chunk size might help. We should followup by auto limit this number when we run gen config

[Bug] Unexpected Error: The model weight size may be larger than GPU memory size

@ahz-r3v you might need to cross check if you have recompiled the lib

[Doc] List of SLM Supported Models

closing as the delivery flow now lands

[Doc] Add info about git and lfs to documentation

added https://github.com/mlc-ai/mlc-llm/pull/2445