Expose vLLM logprobs in model output

[CoolFish88]

trafficstars

Description

vLLM sampling parameters include a richer set of values, among which logprobs has a wider utility.

When testing by adding the logpobs option to the request payload, the model output schema was unchanged {"generated text": "model_output"} suggesting it was not propagated to the output

Will this change the current api? How?

Probably by enriching the output schema.

Who will benefit from this enhancement?

Anyone who wants logprobs extracted from model predictions.

References

• list known implementations This thread provides a starting point for tackling this issue.