gateway
gateway copied to clipboard
bugfix: convert several providers stop_reason to valid openai finish_reason
Description:
Currently, we forward the anthropic stop_reason
, which is not compatible with the openai finish_reason
.
This can cause warnings from some OpenAI sdk.
The conversion function returns stop
as a default value in case new stop_reason
are added.
There was also a bug in the groq and google providers for the finish_reason
Side note: It would probably be beneficial to update the types to we can get compilation-time validation : https://github.com/Portkey-AI/gateway/blob/77f32e57b14bd3585169b7bc884ff9ddf38e8ea1/src/providers/types.ts#L101-L123
Proposed change :
export interface ChatChoice {
index: number;
message: Message;
finish_reason: 'stop' | 'length' | 'function_call' | 'content_filter' | null;
logprobs?: object | null;
}
export interface CompletionResponse extends CResponse {
choices: {
text: string;
index: number;
logprobs: null;
finish_reason: 'stop' | 'length' | 'function_call' | 'content_filter' | null;
}[];
}
see https://platform.openai.com/docs/guides/text-generation/chat-completions-response-format