prompttools
prompttools copied to clipboard
Add OpenAICompatibleChatExperiment class for enhanced LLM compatibility
I have introduced a new class called OpenAICompatibleChatExperiment. This class aims to enhance the compatibility of prompttools with a wider range of large language models (LLMs) that support OpenAI-compatible inference endpoints. This enables usage of popular open source models hosted on Model as API service providers like together.ai, and self hosted models via Huggingface Text Generation Inference/ vLLM/ ollama etc, as they provide openai compatible rest api.
Key features and benefits of the OpenAICompatibleChatExperiment class:
Flexibility: The class allows users to specify the base URL and API key for any OpenAI-compatible inference endpoint, enabling seamless integration with various LLMs. Ease of use: The class maintains a similar interface to the existing OpenAIChatExperiment class, ensuring a smooth transition for users familiar with prompttools. Expanded LLM support: By leveraging OpenAI-compatible endpoints, prompttools can now interface with a broader ecosystem of LLMs, empowering users to explore and utilize different models effortlessly.