Tianqi Chen comments

Results 637 comments of


                                            Tianqi Chen

Chat demo is not working in Chrome 113.

Closing for now, likely latest chrome canary can work

Where stores Web LLM initial weights

this is likely browser dependent, but in your case that looks like the right folder

Where stores Web LLM initial weights

I don't know :) didn't look it up before

Avoid reloading shards in different tabs

webworker is indeed useful, cc @DustinBrett i remember you also mentioned web worker, would love to see if we can collectively build a solution here

Avoid reloading shards in different tabs

webworker support is added in main,I assume that we can adapt this to SharedWorker

Run Web LLM on a Linux on Android?

checkout android examples at https://mlc.ai/mlc-llm/docs/

Whisper in web-llm with WebGPU?

great suggestion, yes this is something that we can push for

data transfer during the execution of Web-LLM

how do you measure data transfer? Besides downloading the initial weights which is cached, WebLLM won't download or upload any data. Maybe you are observing GPU CPU communication as part...

data transfer during the execution of Web-LLM

Right, the initial weights are downloaded and cached. In your case I think it is due to the initial weight download. They are only cached once however, and unless there...

data transfer during the execution of Web-LLM

I think for the case of vicuna it would be 4G. As for model updates, unfortunately the execution code logic can be tied with model versions, so likely that can...