bee-agent-framework icon indicating copy to clipboard operation
bee-agent-framework copied to clipboard
verified publisher icon i-am-bee

Add LMStudio adapter

Open Tomas2D opened this issue 1 year ago • 2 comments

Add LLM inference support for https://lmstudio.ai/

Implementation notes:

  • Use appropriate SDK: https://github.com/lmstudio-ai/lmstudio.js
  • Target directory: src/adapters/lmstudio with files llm.ts and chat.ts.
  • Add tests

Implementation tips:

  • Implementation should be similar to the current Ollama (src/adapters/ollama).

Tomas2D avatar Oct 08 '24 07:10 Tomas2D

Note that it's currently possible to use LM Studio with Bee through the OpenAI-compatible API (https://lmstudio.ai/docs/basics/server#openai-like-api-endpoints), in a similar way to Ollama. I haven't checked if it has limitations over the SDK, however.

JanPokorny avatar Oct 08 '24 12:10 JanPokorny

Then I wonder why their SDK (https://www.npmjs.com/package/@lmstudio/sdk) differs from Ollama / OpenAI SDK.

Tomas2D avatar Oct 08 '24 13:10 Tomas2D

One can use the OpenAI-compatible endpoint. Closing for now.

Tomas2D avatar Mar 20 '25 08:03 Tomas2D