prompttools icon indicating copy to clipboard operation
prompttools copied to clipboard

Add OpenAICompatibleChatExperiment class for enhanced LLM compatibility

Open agyaatcoder opened this issue 9 months ago • 1 comments

I have introduced a new class called OpenAICompatibleChatExperiment. This class aims to enhance the compatibility of prompttools with a wider range of large language models (LLMs) that support OpenAI-compatible inference endpoints. This enables usage of popular open source models hosted on Model as API service providers like together.ai, and self hosted models via Huggingface Text Generation Inference/ vLLM/ ollama etc, as they provide openai compatible rest api.

Key features and benefits of the OpenAICompatibleChatExperiment class:

Flexibility: The class allows users to specify the base URL and API key for any OpenAI-compatible inference endpoint, enabling seamless integration with various LLMs. Ease of use: The class maintains a similar interface to the existing OpenAIChatExperiment class, ensuring a smooth transition for users familiar with prompttools. Expanded LLM support: By leveraging OpenAI-compatible endpoints, prompttools can now interface with a broader ecosystem of LLMs, empowering users to explore and utilize different models effortlessly.

agyaatcoder avatar May 01 '24 14:05 agyaatcoder

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar May 01 '24 14:05 CLAassistant