Explore transformers.js to see what we can leverage

[humphd]

There's a lot of amazing work happening in transformers.js, which lets you run Huggingface Models in the browser (see full docs). We should explore if there are any bits we could use here.

One issue to consider with these models is that you have to download and cache them, so we'd need to have a way to allow the user to opt-in to the downloads. Maybe we provide some kind of "add-on" style UI, where you can enable various features to run locally, and in then we kick off downloading these models to use.

Having these models stored locally would allow us to run some features for free, without doing API calls to remote AI models.

This issue is mainly about exploring this idea, and filing follow-ups to do specific work.

[humphd]

The transformers.js project keeps getting better and better with new ONNX based models being added. Today it's a latex image to text model:

https://twitter.com/xenovacom/status/1746499201184182650?t=Q-jSo3oqjJCmqVa-pXrlSQ

[humphd]

https://twitter.com/xenovacom/status/1753117449749217512?t=Q-jSo3oqjJCmqVa-pXrlSQ is amazing

[humphd]

The Qwen chat models are pretty interesting for running in-browser:

https://github.com/xenova/transformers.js/releases/tag/2.15.0

[humphd]

https://www.youtube.com/watch?v=n18Lrbo8VU8