Rick Zhou comments

Results 6 comments of


                                            Rick Zhou

[KVCache] Support passing in attn_score_scaling_factor into KV cache

cc: @MasterJH5574 Will need https://github.com/flashinfer-ai/flashinfer/pull/126 to be merged first:

API should be exposed to ServiceWorker

@beaufortfrancois @tqchen Thanks a lot for putting the effort to support WebGPU in service worker. I was able to put up a sample Chrome extension running LLM in the service...

[SLM] Support BERT architecture. Implement a text embedding module

> please fix the jenkins here Should be addressed by https://github.com/mlc-ai/mlc-llm/pull/2292. I'm triggering a rebuild now

[SLM] Support BERT architecture. Implement a text embedding module

To fix CUDA error, https://github.com/apache/tvm/pull/16982

[Feature Request] Change OpenAI protocol default value to NOT_GIVEN

https://github.com/mlc-ai/mlc-llm/pull/2178

[Feature Request] Change OpenAI protocol default value to NOT_GIVEN

@tqchen PR to change this in JSONFFIEngine: https://github.com/mlc-ai/mlc-llm/pull/2225