web-llm
web-llm copied to clipboard
mlc-ai
High-performance In-browser LLM Inference Engine
I encountered an error after compiling a WebAssembly (Wasm) format using mlc-llm. The error message I received is as follows: Init error, LinkError: WebAssembly.instantiate(): Import #2 module="env" function="_ZN3tvm7runtime15NVTXScopedRangeD1Ev" error: function...
I have updated to latest version and my models are throwing this error: ``` Init error, Error: Cannot find global function vm.builtin.apply_presence_and_frequency_penalty ``` Last known version that worked for me...
I tried out the [chrome-extension](https://github.com/mlc-ai/web-llm/tree/main/examples/chrome-extension) example as-is and I got errors that the [model](https://huggingface.co/mlc-ai/Mistral-7B-Instruct-v0.2-q4f16_1-MLC/resolve/main/) does not exist. If you click on the url, it indeed does not exist. I replace...
Contributing here requires a bit of extra effort. One improvement would be simply publish the TVMjs package and/or the mlc-ai/relax in npm and then change [this line](https://github.com/mlc-ai/web-llm/blob/3319d1c5a93f57425b964c6b3b13e71dcd349d75/package.json#L40) For seasoned developers...
I'm very impressed of the work made by **web-llm** team and really wanted to try to integrate it into an existing project. Unfortunately current models that capable to do something...
Is this possible to use this pipeline to search locally offline inside the system from specific folders and paths?? Is the setup deployment possible? Please if you can guide me.
For projects that don't use NPM, Is it possible to create a Javascript version that is deployed on a CDN? This will make it easier to integrate with ASP.NET projects...
I'm trying to use Web-LLM with a model I got from HuggingFace and created a WASM version of. I want to load that model from the same webserver that the...
When I try to run a custom model I see these warnings in the browser console: ``` WebSocket connection to 'wss://localhost.dd/' failed: runtime-cd9f0e112c595857.js:174 [parcel] 🚨 Connection to the HMR server...
Having tested StableLM1.6 successfully! 🎉 The model has a 4096 context and its was constantly crashing due to memory errors but I was thinking that I had other problems. We...