julep icon indicating copy to clipboard operation
julep copied to clipboard

[Bug]: Remove support for `openai.chat.completions.create(..., n>1)` everywhere

Open creatorrr opened this issue 1 year ago • 1 comments
trafficstars

📜 Description

So the problem is that if you use n>1, api will return more than one result in the {"choices": [...]} but then:

  1. how do you unwrap?
  2. if there's a tool-call, how do you handle it as in which choice's tool-call should you use?
  3. no one really uses choices anymore (reasoning models don't support it either)

Possible solutions

  1. Remove support from typespec and rewrite (simplify) the prompt step logic
  2. Just consider tool-calls with n>1 as wait for input

👀 Have you spent some time to check if this bug has been raised before?

  • [X] I checked and didn't find similar issue

creatorrr avatar Nov 09 '24 10:11 creatorrr

@whiterabbit1983 thoughts?

creatorrr avatar Feb 15 '25 03:02 creatorrr