langchain icon indicating copy to clipboard operation
langchain copied to clipboard

Ungraceful handling of empty completions (i.e., completion that begins with a stop sequence)

Open s16h opened this issue 1 year ago • 0 comments

If the model predicts a completion that begins with a stop sequence, resulting in no output, we'll get a KeyError:

Traceback (most recent call last):
...
  File ".../langchain/llms/openai.py", line 152, in _generate
    token_usage[_key] = response["usage"][_key]
KeyError: 'completion_tokens'

That's because response is an empty dict.

You can reproduce this with the following prompt and params:

response = openai.Completion.create(
  model="text-davinci-003",
  prompt="Remove all words from \"Original Text\".\n\nOriginal Text: yeah.\n\nCorrected Text:",
  temperature=0,
  max_tokens=256,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0
)

s16h avatar Jan 21 '23 09:01 s16h