feat: Add LM Studio as a local model provider alongside Ollama

[hghalebi]

Feature Request

Add LM Studio as a local model provider alongside Ollama

Motivation

LM Studio demonstrates significant performance advantages over Ollama with the same models:

• 26-30% higher tokens/second on identical hardware
• Provides detailed performance metrics (tokens/sec, TTFT) in API responses
• GPU offloading enables partial acceleration when models exceed VRAM
• OpenAI-compatible API at http://localhost:1234/v1/*

For local agent deployments, this performance difference is crucial for real-time responsiveness.

Proposal

Add rig::providers::lmstudio module implementing the CompletionModel trait:

// rig-core/src/providers/lmstudio/mod.rs
pub struct Client {
    base_url: String,
    http_client: reqwest::Client,
}

impl Client {
    pub fn from_default() -> Self {
        Self::new("http://localhost:1234/v1")
    }
}

The provider would:

1. Use OpenAI-compatible endpoints (already proven in Rig)
2. Support streaming responses
3. Expose LM Studio's performance metrics in responses

[linear[bot]]

RIG-902 feat: Add LM Studio as a local model provider alongside Ollama

[rajiknows]

👍 on it .

[joshua-mo-143]

Before we fully commit to this, is it possible that we can just add #[serde(flatten)] on a serde_json::Map to the Message::Assistant variant of the OpenAI completion model and call it a day?

The reason I ask is that this seems like a lot of work for something that we could realistically add at almost zero cost to the OpenAI Chat Completions API integration. This particular integration seems to already work with LM Studio and I would be willing to bet there are also other local-first LLM programs that could probably benefit.

[rajiknows]

Since it’s OpenAI-compatible, your solution works perfectly, @joshua-mo-143 . We’ll just add a wrapper around OpenAI provider in lmstudio.rs and be done with it

[Abel2333]

Before we fully commit to this, is it possible that we can just add #[serde(flatten)] on a serde_json::Map to the Message::Assistant variant of the OpenAI completion model and call it a day?

The reason I ask is that this seems like a lot of work for something that we could realistically add at almost zero cost to the OpenAI Chat Completions API integration. This particular integration seems to already work with LM Studio and I would be willing to bet there are also other local-first LLM programs that could probably benefit.

But i found that if i use rig::client::providers::openai to build an agent, it wound only send the request to the http://localhost:1234/v1/response, rather than http://localhost:1234/v1/chat/completions. How can we use the OpenAI Compatible API?

[joshua-mo-143]

Before we fully commit to this, is it possible that we can just add #[serde(flatten)] on a serde_json::Map to the Message::Assistant variant of the OpenAI completion model and call it a day? The reason I ask is that this seems like a lot of work for something that we could realistically add at almost zero cost to the OpenAI Chat Completions API integration. This particular integration seems to already work with LM Studio and I would be willing to bet there are also other local-first LLM programs that could probably benefit.

But i found that if i use rig::client::providers::openai to build an agent, it wound only send the request to the http://localhost:1234/v1/response, rather than http://localhost:1234/v1/chat/completions. How can we use the OpenAI Compatible API?

The OpenAI Responses completion model has a method to revert back to the original completion model:

let openai_client = rig::providers::openai::Client::from_env();

let completions_api_model = openai_client.completion_model("gpt-4o")
    .completions_api()
    .into_agent_builder(); // turn it into an agent if you want!

The reason why the Responses API is the default is because, simply speaking, it is more performant/better. The OpenAI team have pointed this out themselves: link

[Abel2333]

Before we fully commit to this, is it possible that we can just add #[serde(flatten)] on a serde_json::Map to the Message::Assistant variant of the OpenAI completion model and call it a day? The reason I ask is that this seems like a lot of work for something that we could realistically add at almost zero cost to the OpenAI Chat Completions API integration. This particular integration seems to already work with LM Studio and I would be willing to bet there are also other local-first LLM programs that could probably benefit.

But i found that if i use rig::client::providers::openai to build an agent, it wound only send the request to the http://localhost:1234/v1/response, rather than http://localhost:1234/v1/chat/completions. How can we use the OpenAI Compatible API?

The OpenAI Responses completion model has a method to revert back to the original completion model:

let openai_client = rig::providers::openai::Client::from_env();

let completions_api_model = openai_client.completion_model("gpt-4o") .completions_api() .into_agent_builder(); // turn it into an agent if you want!

The reason why the Responses API is the default is because, simply speaking, it is more performant/better. The OpenAI team have pointed this out themselves: link

Thank you, that make my program more useful.

[katopz]

a bit confuse here, do i need to wait for the PR to merge or use openai workaround?

[joshua-mo-143]

a bit confuse here, do i need to wait for the PR to merge or use openai workaround?

At the moment, you can already use LM Studio as-is through switching back to OpenAI Completions API.

There's an infinite amount of providers that are compliant with the OpenAI Completions API, so adding them all individually will be quite a maintenance burden 😅

[katopz]

Thanks, that's understandable. Anyhow this (working code) took me sometime to figure this out 😅 so maybe this should be add to the docs somewhere.

    let client = Client::builder("")
        .base_url("http://localhost:1234/v1")
        .build()?;

    let agent = client
        .completion_model(model_name)
        .completions_api()
        .into_agent_builder()
        .preamble(SYSTEM_PREAMBLE)
        .tool(SolTransferTool)
        .build();

[joshua-mo-143]

Hi folks,

As there is nothing to actually fix/add here (and there are currently no plans to accept a PR that will fix this issue) due to the workaround, we will be closing this issue.