rig icon indicating copy to clipboard operation
rig copied to clipboard

feat: Update completion API `Message` type

Open cvauclair opened this issue 2 months ago • 1 comments

  • [x] I have looked for existing issues (including closed) about this

Feature Request

Update the completion::Message type to reflect new features in popular model providers.

Note: this would solve #121 and offer limited multimodal support in the case where text completion models accept images as input.

Motivation

Many popular completion model providers have expanded their APIs to support different types of message. For instance, OpenAI now supports the tool message type and has added the ability to upload images using the user message type. Anthropic also supports uploading images using the user message type.

Supporting these advanced features would make Rig a lot more flexible.

Here is a (non-exhaustive) list of providers and the message types we currently do not support:

  • OpenAI: tool messages, tool_calls in assistant messages and images in user messages
  • Anthropic: images in user messages and ToolResult as a message content type
  • Cohere: tool message and tool_calls in assistant messages

Proposal

  1. Refactor the completion::Message type such that is more complete, e.g. using an enum:

    pub enum Message {
        UserMessage(...),
        AssistantMessage(...),
        SystemMessage(...),
        ToolMessage(...),
    }
    

    Note: Make sure that the new Message type covers all possible types across all providers.

  2. Update the provider client implementations to covert the new Message type to their own completion model.

    Important: It is important to handle the case where the provider does not support a given message type (e.g.: Anthropic does not support the tool message type). There are two options to handle those cases: 1) return an error; 2) convert them to another type that makes sense (e.g.: converting the ToolMessage to a user type message for Anthropic). Not all types can be converted though. For instance, if a provider does not support an image user message, then the client should return an error.

Alternatives

An alternative could be to add an additional_params field to the existing Message type allowing users to provide additional parameters on a per-provider basis, similar to how additional_params is available on CompletionRequest.

cvauclair avatar Dec 09 '24 23:12 cvauclair