langchain
langchain copied to clipboard
Ungraceful handling of empty completions (i.e., completion that begins with a stop sequence)
If the model predicts a completion that begins with a stop sequence, resulting in no output, we'll get a KeyError
:
Traceback (most recent call last):
...
File ".../langchain/llms/openai.py", line 152, in _generate
token_usage[_key] = response["usage"][_key]
KeyError: 'completion_tokens'
That's because response
is an empty dict.
You can reproduce this with the following prompt and params:
response = openai.Completion.create(
model="text-davinci-003",
prompt="Remove all words from \"Original Text\".\n\nOriginal Text: yeah.\n\nCorrected Text:",
temperature=0,
max_tokens=256,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)