gateway icon indicating copy to clipboard operation
gateway copied to clipboard

bugfix: convert several providers stop_reason to valid openai finish_reason

Open unsync opened this issue 1 month ago • 0 comments

Description: Currently, we forward the anthropic stop_reason, which is not compatible with the openai finish_reason.

This can cause warnings from some OpenAI sdk.

The conversion function returns stop as a default value in case new stop_reason are added.

There was also a bug in the groq and google providers for the finish_reason

Side note: It would probably be beneficial to update the types to we can get compilation-time validation : https://github.com/Portkey-AI/gateway/blob/77f32e57b14bd3585169b7bc884ff9ddf38e8ea1/src/providers/types.ts#L101-L123

Proposed change :

export interface ChatChoice {
  index: number;
  message: Message;
  finish_reason: 'stop' | 'length' | 'function_call' | 'content_filter' | null;
  logprobs?: object | null;
}
export interface CompletionResponse extends CResponse {
  choices: {
    text: string;
    index: number;
    logprobs: null;
    finish_reason: 'stop' | 'length' | 'function_call' | 'content_filter' | null;
  }[];
}

see https://platform.openai.com/docs/guides/text-generation/chat-completions-response-format

unsync avatar May 14 '24 09:05 unsync