Stream with tool call and langchain fail

[alejandrojnm]

I write a simple api to test the lib using Llama (vllm) as backend, everything work fine if I don't send a tool. I'm sure is not the model because I test with https://github.com/sashabaranov/go-openai and work fine.

I'm making the call using langchain

this Is the code that I have, can be something I have wrong ?

body, err := io.ReadAll(r.Body)
	if err != nil {
		utils.JSONError(w, http.StatusBadRequest, "Invalid request body")
		return
	}

	var chatRequest openai.ChatCompletionNewParams
	err = json.Unmarshal(body, &chatRequest)
	if err != nil {
		utils.JSONError(w, http.StatusBadRequest, "Invalid request body")
		return
	}

// Create the stream
		stream := client.Chat.Completions.NewStreaming(
			r.Context(),
			chatRequest,
		)

		acc := openai.ChatCompletionAccumulator{}
		for stream.Next() {
			chunk := stream.Current()
			acc.AddChunk(chunk)

			if len(chunk.Choices) > 0 {
				utils.SendEvent(w, utils.EventStream{
					Data: chunk,
				})
			}
		}

any idea why ?

[jacobzim-stl]

Thanks for this report, I'll look into this.

[alejandrojnm]

I'm using net/http and mux, in case that help you to debug the problem

[jacobzim-stl]

I wasn't able to reproduce this bug, would you mind sharing a request.

You can a request with option.WithDebugLog(log.Default)

[alejandrojnm]

I will try again with the last version of the lib and that option.