eino icon indicating copy to clipboard operation
eino copied to clipboard
Published 1 week ago • verified publisher icon cloudwego

Litellm open ai compatible protocol

Open ngoiyaeric opened this issue 10 months ago • 4 comments

—> feat: A new feature optimize: A new optimization fix: A bug fix perf: A code change that improves performance -->

Check the PR title.

  • [ ] This PR title match the format: <type>(optional scope): <description>
  • [ ] The description of this PR title is user-oriented and clear enough for others to understand.
  • [ ] Attach the PR updating the user documentation if the current PR requires user awareness at the usage level. [User docs repo](https://github.com/

(Optional) More detailed description for this PR(en: English/zh: Chinese).

en: zh(optional):

(Optional) Which issue(s) this PR fixes:

Add LitellmModel for OpenAI-compatible API integration.

Define LitellmModel struct with a client field for OpenAI-compatible APIs. Implement NewLitellmModel function to create a new instance of LitellmModel. Add GenerateText method to generate text based on a given prompt using the OpenAI-compatible API. Include a comment explaining Litellm's compatibility with the OpenAI API protocol.

ngoiyaeric avatar Mar 08 '25 15:03 ngoiyaeric

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Mar 08 '25 15:03 CLAassistant

Hi, thanks for contribution.

  • This repository is used to store component abstractions, orchestration capabilities, Agent paradigms, and other implementation-independent code. Component implementations can be submitted in the eino-ext.
  • Components in eino-ext must implement interface defined in eino/component, for example, https://github.com/cloudwego/eino/blob/df93610a63a7d30d9cccddffe313d4ee967dccbf/components/model/interface.go#L29
  • Why do you need this pr? I noticed that this protocol is exactly the same as OpenAI's, so maybe you can directly use the OpenAI package?

meguminnnnnnnnn avatar Mar 10 '25 12:03 meguminnnnnnnnn

Do you have any code specific improvements should I be calling https://github.com/openai/openai-go from instead? Should this be implemented on eino-ext in this package eino-ext/libs/acl as a litellm folder with open ai protocol

ngoiyaeric avatar Mar 11 '25 11:03 ngoiyaeric

It seems that this PR simply wraps the OpenAI protocol, which has already been done in https://github.com/cloudwego/eino-ext/tree/main/components/model/openai.

meguminnnnnnnnn avatar Mar 11 '25 11:03 meguminnnnnnnnn