Embeddings with web-llm?

[singerbj]

trafficstars

I am generally a n00b when it comes to LLMs and data science in general so apologies if this question doesn't make sense:

Is it possible to use embeddings with web-llm? Is there any guide for how to do so?

[CharlieFRuan]

Thanks for the question! It is possible and is on the to-do list for us. As of now, the project is still LLM-oriented, but we are currently working on expanding the features we support.

[matthoffner]

Hello, I've built a simple demo linked in this project that uses transformers.js embeddings for a simple vector store search: https://huggingface.co/spaces/matthoffner/web-llm-embed/blob/main/src/components/ChatWindow.jsx#L40-L66

The full space is linked in the examples: https://github.com/mlc-ai/web-llm/tree/main/examples

Hope this helps!

[ggaabe]

@matthoffner this example doesn't seem to use the mlc-ai/web-llm library at all?

[matthoffner]

@ggaabe True, I believe it was built before the library was available. It uses react-llm which uses the web-llm model.

[CharlieFRuan]

Hi @ggaabe @matthoffner @singerbj, npm ^0.2.60 has initial support for embedding and RAG, check out usage here: https://github.com/mlc-ai/web-llm/tree/main/examples/embeddings

Currently only snowflake-arctic-embedding is supported.

Closing for now, feel free to open new issues.