`Instructor.echo_response/1` no function clause matching

[kevinschweikert]

We are using Version 0.0.5 in our application. We recently got this error:

** (FunctionClauseError) no function clause matching in Instructor.echo_response/1
(instructor 0.0.5) lib/instructor.ex:510: Instructor.echo_response(%{"choices" => [%{"finish_reason" => "stop", "index" => 0, "message" => %{"content" => "[OUR_CONTENT]", "role" => "assistant"}}], "created" => 1709884788, "id" => "chatcmpl-90PQq8TYhzCvxtCz2w2xneKudwU0O", "model" => "gpt-4-1106-vision-preview", "object" => "chat.completion", "usage" => %{"completion_tokens" => 76, "prompt_tokens" => 1408, "total_tokens" => 1484}})
(instructor 0.0.5) lib/instructor.ex:456: anonymous fn/3 in Instructor.do_chat_completion/3

I couldn't find time to look at the code yet. Maybe I could provide a fix for this later.

[thmsmlr]

Does your code also use validations and max_retries? Could be a weird interaction between the vision api response and the instructor validation / reasking feature

[kevinschweikert]

Yes, this is the code we're using:

 Instructor.chat_completion(
  model: "gpt-4-vision-preview",
  response_model: __MODULE__,
  max_tokens: 800,
  max_retries: 3,
  mode: :md_json,
  messages: [
    %{
      role: "user",
      content: [
        %{type: "text", text: data.image_to_text_prompt},
        %{type: "image_url", image_url: %{url: data.original_image, detail: "high"}}
      ]
    },
    %{
      role: "system",
      content: "Straightforward and direct answers."
    }
  ]
)

And we're doing validation with Ecto.Changesets

[dankentfield]

I had the same issue here too

** (FunctionClauseError) no function clause matching in Instructor.echo_response/1    
    
    The following arguments were given to Instructor.echo_response/1:
    
        # 1
        %{
          "choices" => [
            %{
              "finish_reason" => "stop",
              "index" => 0,
              "logprobs" => nil,
              "message" => %{
                "content" => "blah blah blah",
                "role" => "assistant"
              }
            }
          ],
          "created" => 1715288065,
          "id" => "chatcmpl-9N54Tf0WXPiYPCkrWq5PbiKLvGAa2",
          "model" => "gpt-4-turbo-2024-04-09",
          "object" => "chat.completion",
          "system_fingerprint" => "fp_0737e0dfd9",
          "usage" => %{
            "completion_tokens" => 419,
            "prompt_tokens" => 1420,
            "total_tokens" => 1839
          }
        }
    
    Attempted function clauses (showing 1 out of 1):
    
        defp echo_response(%{
      "choices" => [
        %{
          "message" =>
            %{
              "tool_calls" => [
                %{"id" => tool_call_id, "function" => %{"name" => name, "arguments" => args}} =
                  function
              ]
            } = message
        }
      ]
    })
    
    (instructor 0.0.5) lib/instructor.ex:510: Instructor.echo_response/1
    (instructor 0.0.5) lib/instructor.ex:456: anonymous fn/3 in Instructor.do_chat_completion/3
    (elixir 1.16.2) lib/keyword.ex:1149: Keyword.update_guarded/4
    (elixir 1.16.2) lib/keyword.ex:1153: Keyword.update_guarded/4
    (instructor 0.0.5) lib/instructor.ex:454: Instructor.do_chat_completion/3
    iex:14: (file)
iex(14)> 

Looks like tool_calls is for function calling, so not entirely sure why it would be expected in a vision response. Not an expert here so take it with a pinch of salt.

[srcrip]

I am getting this issue as well. I'm not using vision though. Just sending regular text. I seem to be getting this with both the chat gpt and ollama adapters. And I'm using the json mode, while this seems related to tool calling so I don't really get what's happening

[harmonicss]

We have hit this problem too, and appreciate the fix. Is there any chance this can be merged back to main?

[thmsmlr]

Can someone on this thread verify that they're still getting this issue with the latest release? We did an internal refactor that got rid of this function and put the responsibility on the specific adapters to implement re-asking.