logprobs exposed in chat completion endpoint

[whitead]

Describe the feature or improvement you're requesting

We found it useful to have logprobs for completions to evaluate confidence. We would like to have access to this attribute in the new chat completion endpoint. Thank you!

Additional context

No response

[sheikheddy]

Yep. Thought I had a workaround but sadly this doesn't work:

openai.Completion.create(model="gpt-3.5-turbo", prompt="Gimme logprobs", max_tokens=5, logprobs=3)

Here's what it gives you:

InvalidRequestError: This is a chat model and not supported in the v1/completions endpoint. Did you mean to use v1/chat/completions?

Could really use logprobs in chat completions.

[primeviking]

^ this! Is this on the roadmap or is there a workaround?

[logankilpatrick]

Hey folks, we are aware people are asking for this, no direct timeline for it but something we are keeping in mind.

[logankilpatrick]

If this is something you are looking for, please drop a 👍 above so the team can get a sense of demand.

[logankilpatrick]

I am going to close for now since this repo is for the Python SDK not API feedback but will follow up if we end up adding this.

[tiangolo]

Thanks @logankilpatrick !

What would be the right place to submit API feedback? Or what would be the place to subscribe to/follow to know about this? ...Or would it be just this same issue and there's no need to check somewhere else?

[pradeepdev-1995]

@tiangolo any update on this? still i am getting the error openai.error.InvalidRequestError: Unrecognized request argument supplied: logprobs when trying to configure logprobs = 0 in openai.ChatCompletion.create

code i tried

completion = openai.ChatCompletion.create(
            engine=gpt-3.5-turbo,
            temperature=0.2,
            messages=[{"role": "user", "content": prompt}],
            logprobs = 0
        )