[Bug]: Azure OpenAI gpt-4 version turbo-2024-04-09 responses are cut off for vision queries

[illgitthat]

What happened?

I recently upgraded the gpt-4 model to turbo-2024-04-09. This is a multimodal model available in Azure.

It looks like vision responses in particular are getting cut off.

Steps to Reproduce

1. Deploy relevant azure resource

    groups:
      - group: "instancename"
        apiKey: "${EAST2_API_KEY}"
        instanceName: "instancename"
        version: "2024-03-01-preview"
        models:
          gpt-4:
            deploymentName: gpt-4 # turbo-2024-04-09, 70k TPM
            version: "2024-03-01-preview" # also tried 2024-02-15-preview which matches what is used in the playground

2. Compare image responses in librechat vs. chat playground.

What browsers are you seeing the problem on?

No response

Relevant log output

2024-05-02T18:25:48.885Z debug: [BaseClient] Loading history:
{
  conversationId: "809e7cf2-108a-4145-9786-4793fe347742",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
}
2024-05-02T18:25:49.464Z debug: [BaseClient] Context Count (1/2)
{
  remainingContextTokens: 7408,
  maxContextTokens: 8187,
}
2024-05-02T18:25:49.466Z debug: [BaseClient] Context Count (2/2)
{
  remainingContextTokens: 7408,
  maxContextTokens: 8187,
}
2024-05-02T18:25:49.466Z debug: [BaseClient] tokenCountMap:
{
  ce7b73ac-8a6c-4afb-b428-3c5eb155e8a7: 776,
}
2024-05-02T18:25:49.468Z debug: [BaseClient]
{
  promptTokens: 779,
  remainingContextTokens: 7408,
  payloadSize: 1,
  maxContextTokens: 8187,
}
2024-05-02T18:25:49.469Z debug: [BaseClient] tokenCountMap
{
  ce7b73ac-8a6c-4afb-b428-3c5eb155e8a7: 776,
  instructions: undefined,
}
2024-05-02T18:25:49.502Z debug: [BaseClient] userMessage
{
  messageId: "ce7b73ac-8a6c-4afb-b428-3c5eb155e8a7",
  parentMessageId: "00000000-0000-0000-0000-000000000000",
  conversationId: "809e7cf2-108a-4145-9786-4793fe347742",
  sender: "User",
  text: "Describe this image in great detail.",
  isCreatedByUser: true,
  // 1 image_url(s)
  image_urls: [{"type":"image_url","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAwAAAAP/CAIAAA... [truncated]],
  tokenCount: 776,
}
2024-05-02T18:25:51.450Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "https://instancename.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=202... [truncated]",
    modelOptions.model: "gpt-4",
    modelOptions.temperature: 1,
    modelOptions.top_p: 1,
    modelOptions.presence_penalty: 0,
    modelOptions.frequency_penalty: 0,
    modelOptions.stop: undefined,
    modelOptions.user: "660ce7e4eca51f37c251afe0",
    modelOptions.stream: true,
    // 1 message(s)
    modelOptions.messages: [{"role":"user","content":[{"type":"text","text":"Describe this image in great detail."},{"type":"ima... [truncated]],
}
2024-05-02T18:25:56.872Z debug: [OpenAIClient] chatCompletion response
{
  object: "chat.completion",
  // 2 prompt_filter_result(s)
  prompt_filter_results: [{"prompt_index":0,"content_filter_result":{"jailbreak":{"filtered":false,"detected":false},"custom_b... [truncated],{"prompt_index":1,"content_filter_result":{"sexual":{"filtered":false,"severity":"safe"},"violence":... [truncated]],
  id: "chatcmpl-9KVPv1j18ZTCYZ8VsMh1jsg9wbcV7",
  // 1 choice(s)
  choices: [{"content_filter_results":{},"message":{"role":"assistant","content":"This image captures a tranquil... [truncated]],
  created: 1714674355,
  model: "gpt-4-turbo-2024-04-09",
}
2024-05-02T18:25:56.874Z debug: [spendTokens] conversationId: 809e7cf2-108a-4145-9786-4793fe347742 | Context: message | Token usage:
{
  promptTokens: 779,
  completionTokens: 16,
}
2024-05-02T18:25:57.383Z debug: [AskController] Request closed
2024-05-02T18:25:57.461Z debug: [OpenAIClient] chatCompletion
{
  baseURL: "https://openai-gpt4-canada-instance.openai.azure.com/openai/deployments/gpt-35-turbo/chat/completion... [truncated]",
    modelOptions.model: "gpt-35-turbo",
    modelOptions.temperature: 0.2,
    modelOptions.top_p: 1,
    modelOptions.presence_penalty: 0,
    modelOptions.frequency_penalty: 0,
    modelOptions.stop: undefined,
    modelOptions.user: "660ce7e4eca51f37c251afe0",
    modelOptions.max_tokens: 16,
    // 1 message(s)
    modelOptions.messages: [{"role":"system","content":"Please generate a concise, 5-word-or-less title for the conversation, us... [truncated]],
}
2024-05-02T18:25:58.605Z debug: [OpenAIClient] chatCompletion response
{
  // 1 choice(s)
  choices: [{"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":{"filtered":false... [truncated]],
  created: 1714674358,
  id: "chatcmpl-9KVPyWhpjQ4q9ymQm4UgHUFi2hKJt",
  model: "gpt-35-turbo",
  object: "chat.completion",
  // 1 prompt_filter_result(s)
  prompt_filter_results: [{"prompt_index":0,"content_filter_results":{"hate":{"filtered":false,"severity":"safe"},"self_harm":... [truncated]],
  system_fingerprint: "fp_2f57f81c11",
    usage.completion_tokens: 8,
    usage.prompt_tokens: 101,
    usage.total_tokens: 109,
}
2024-05-02T18:25:58.606Z debug: [spendTokens] conversationId: 809e7cf2-108a-4145-9786-4793fe347742 | Context: title | Token usage:
{
  promptTokens: 98,
  completionTokens: 8,
}
2024-05-02T18:25:58.608Z debug: [OpenAIClient] Convo Title: Tranquil Night Scene with Palm Trees

Screenshots

Code of Conduct

• [X] I agree to follow this project's Code of Conduct