VCs: EU-hosted LLM

[SimoneZaza]

Description

Brief summary of what this Epic is, whether it's a larger project, goal, or user story. Describe the job to be done, which persona this Epic is mainly for, or if more multiple, break it down by user and job story.

Goal

Describe how this Epic impacts an initiative the business is working on.

Hypothesis

What is your hypothesis on the success of this Epic? Describe how success will be measured and what leading indicators the team will have to know if success has been hit.

Must have scope

Define what is a must-have for launch and in-scope. Keep this section fluid and dynamic until you lock-in priority during planning.

Stakeholders

Describe who needs to be kept up-to-date about this Epic, included in discussions, or updated along the way. Stakeholders can be both in Product/Engineering, as well as other teams like Customer Success who might want to keep customers updated on the Epic project.

From duplicate epic

Description

We need alternatives to the LLM, ideally open source and cost-effective.

Must have scope

• [ ] An EU-based LLM, running with similar responses to the OpenAI's one.
• [ ] Revise the embeddings model (detach from OpenAI).

Additional info: LLMs to test:

• Mistral (open and closed)
• Groq (LPU)

Unknown - LLM response quality? Limitations - language support?

https://docs.mistral.ai/getting-started/models/