rig
rig copied to clipboard
feat: Update completion API `Message` type
- [x] I have looked for existing issues (including closed) about this
Feature Request
Update the completion::Message
type to reflect new features in popular model providers.
Note: this would solve #121 and offer limited multimodal support in the case where text completion models accept images as input.
Motivation
Many popular completion model providers have expanded their APIs to support different types of message. For instance, OpenAI now supports the tool
message type and has added the ability to upload images using the user
message type. Anthropic also supports uploading images using the user
message type.
Supporting these advanced features would make Rig a lot more flexible.
Here is a (non-exhaustive) list of providers and the message types we currently do not support:
- OpenAI:
tool
messages,tool_calls
inassistant
messages and images inuser
messages - Anthropic: images in
user
messages andToolResult
as a message content type - Cohere:
tool
message andtool_calls
inassistant
messages
Proposal
-
Refactor the
completion::Message
type such that is more complete, e.g. using an enum:pub enum Message { UserMessage(...), AssistantMessage(...), SystemMessage(...), ToolMessage(...), }
Note: Make sure that the new
Message
type covers all possible types across all providers. -
Update the provider client implementations to covert the new
Message
type to their own completion model.❗ Important: It is important to handle the case where the provider does not support a given message type (e.g.: Anthropic does not support the
tool
message type). There are two options to handle those cases: 1) return an error; 2) convert them to another type that makes sense (e.g.: converting theToolMessage
to auser
type message for Anthropic). Not all types can be converted though. For instance, if a provider does not support an image user message, then the client should return an error.
Alternatives
An alternative could be to add an additional_params
field to the existing Message
type allowing users to provide additional parameters on a per-provider basis, similar to how additional_params
is available on CompletionRequest
.