Tianqi Chen comments

Results 637 comments of


                                            Tianqi Chen

Chat demo does not work on Android because of maxStorageBufferBindingSize

The error log indeed indicate possible OOM ("gpu get lost") because the model is too VRAM demanding

Chat demo does not work on Android because of maxStorageBufferBindingSize

We need to know the total amount of VRAM that can be allocated. In Vulkan I think may corresponds to `PhysicalDeviceMemoryProperties ` and memory heap size. Note that there can...

Next example doesn't work unless source is built

Get it.. this is surprising, would be useful to look into the difference. I wonder if it has to do with way things are bundled

Chat demo does not work on Android because of maxStorageBufferBindingSize

I think in this case, likely reacting to WebLLM might work better, then leading to a restart of the UI. https://github.com/mlc-ai/web-llm/blob/main/examples/simple-chat/src/simple_chat.ts#L239 cc @CharlieFRuan do you mind to take a look

simple chat example "npm start"

thanks @tlopex do you mind send a PR

Change the system message

this is now supported

Compiling web-llm into a wasm file for improved language interoperability and a sample project inclusion request

https://github.com/mlc-ai/mlc-llm/issues/2218

Langchain integration

langchain js now have webllm integration

Could make it more clear on pre-requiest

please checkout the latest instructions in https://mlc.ai/mlc-llm/docs/compilation/compile_models.html

document is not defined in Web Workers

Thanks @DustinBrett , do you mind to send a PR to fix this