leapfrogai
leapfrogai copied to clipboard
Return finish reason field & VLLM Refactor
When performing a completion or chat completion return the finish_reason according to the api spec.
- Adds finish reason in completion/chat completion that mirrors the openai spec
- Adds token usage in completion/chat completion responses that mirrors the openai spec
Closes https://github.com/defenseunicorns/leapfrogai/issues/463 Closes https://github.com/defenseunicorns/leapfrogai/issues/431
Deploy Preview for leapfrogai-docs canceled.
| Name | Link |
|---|---|
| Latest commit | 99661575447dd4871a298155cf60c73dde288cc0 |
| Latest deploy log | https://app.netlify.com/sites/leapfrogai-docs/deploys/664cd436bf4db50008143d93 |
Conforming to the specs for token length will require updating all of the backends. It assumes that there's the token counting function. TBD if doing that in this commit.