Best small model for text generation?

[iamhenry]

Question

I'm looking to build a AI Journaling app that helps you reflect from your journal entries

I'm looking for a model like (GPT or Claude) that will take the selected text and provide insights based on a prompt I provide

In this case the prompt will provide suggestions based on psychology techniques like CBT and ACT to help you with your life.

Any ideas on which small model will be able to accomplish this? I've tried GPT2, t5- small, and I couldn't get Phi-3 to work

[flatsiedatsie]

I'd really suggest Phi 3, which Transformers.js can run with GPU acceleration.

I'm looking for a model like (GPT or Claude)

Curb your expectations.

You might also want to look into WebLLM and Wllama for running text-based models.

[iamhenry]

@flatsiedatsie thx. I'm looking at the new qwen 2.5 but running into issue

1. cant locate file via URL
2. when i download the file to my laptop (after trying to intialize model) Error: Failed to load model with error: /Users/runner/work/1/s/onnxruntime/core/graph/model.cc:146 onnxruntime::Model::Model(onnx::ModelProto &&, const onnxruntime::PathString &, const onnxruntime::IOnnxRuntimeOpSchemaRegistryList *, const logging::Logger &, const onnxruntime::ModelOptions &) Unsupported model IR version: 10, max supported IR version: 8

created a ticket here: https://github.com/xenova/transformers.js/issues/944

[flatsiedatsie]

You might want to take a look at this: https://huggingface.co/spaces/webml-community/llama-3.2-webgpu

[iamhenry]

@flatsiedatsie whoa, thx!

how is webml different than transformers.js?

[flatsiedatsie]

it's not different.