[RFC]: Deprecate stop_reason in OpenAI Entrypoint in favor of finish_reason; fix implementation of finish_reason

[K-Mistele]

Motivation.

This RFC is motivated by a desire to ensure full OpenAI compatibility in Completions & chat completions endpoints by deprecating stop_reason which is not part of the OpenAI specification, and fixing the finish_reason field's implementation to be OpenAI-compatibile even though doing so may (or may not) be a breaking change for a small number of existing users.

Proposed Change.

Suggestion: recommend deprecating stop_reason

The Chat Completion Object in OpenAI's Documentation indicates a finish_reason field, but not a stop_reason. While the stop_reason doesn't break the OpenAI compatibility since it's just an extra property, it is a source of confusion for contributors such as myself, as well as implementers & users of vLLM.

It is unclear when stop_reason was introduced to the project or why, although I am sure this could be determined by going back through old issues. Perhaps it is an artifact of an older version of OpenAI, or it was merely a typo?

Currently, stop_reason is almost always null/None - it does not seem to add any value to the project.

Please share your feedback on deprecating and removing the stop_reason field from completion & chat completion responses in the OpenAI-compatible entrypoint :)

Suggestion: change finish_reason behavior for OpenAI compatibility

Currently, the finish_reason field in the completion & chat completion responses of the OpenAI-compatible entrypoint is almost always null/None (RESTful API vs. Python API). Per the OpenAI specification for the chat completion object, the finish_reason should be one of the following:

• length if the response is terminated early due to a requested maximum length
• stop if the model stops naturally or a stop sequence is reached
• tool_calls if the model generates tool calls
• content_filter if the content filter terminates the model's response (not applicable for us)
• function_calls deprecated by tool_calls and therefore not applicable to us

Per the OpenAI specification, finish_reason should never be null or None

My recommendation is to fix the implementation of finish_reason in these endpoints to ensure maximum OpenAI compatibility. PLEASE NOTE that it is possible that this change may be breaking for vLLM users who are using the OpenAI-compatible endpoint with non-OpenAI tooling and who are relying on a None/null value in the finish_reason field, but this change is required for full OpenAI compatibility.

Feedback Period.

I am attempting to finish implementing OpenAI-style tool calling in #5649 and would like to implement this fix in that PR, particularly by implementing the tool_calls value in the finish_reason field where appropriate. I anticipate that PR being ready for review in 1-2 weeks.

I would ideally like to have community and maintainer feedback by then, so that I can implement this fix/change and so that it doesn't become a blocker on that PR.

CC List.

@mgoin @WoosukKwon @simon-mo

Any Other Things.

This RFC is pertinent to PR #5649 as the tool_call value for the finish_reason needs to be implemented, so i would like to properly implement the stop value as well.

[ywang96]

Thank you for this RFC - imo we should support OpenAI compatibility at our best effort so working on finished_reason aligns with our goal.

I do think there's a value of keeping stop_reason since finished_reason can't really be a direct drop-in replacement, especially if some users already rely on for their downstream tasks.

[simon-mo]

Adding @njhill who initially added this.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this issue should remain open. Thank you!

[github-actions[bot]]

This issue has been automatically closed due to inactivity. Please feel free to reopen if you feel it is still relevant. Thank you!