FastChat icon indicating copy to clipboard operation
FastChat copied to clipboard

Please support top_logprobs via API

Open xufengduan opened this issue 1 year ago • 0 comments

I've noticed that the current implementation supports logprobs (thanks a lot!) but it appears to be different from how OpenAI handles them. I was wondering if it would be possible to enhance the functionality in the following ways:

  1. Support for top_logprobs: This would allow users to retrieve the log probabilities for the top tokens directly, which can be immensely useful for applications requiring more nuanced probability assessments.
  2. API Endpoint for Specific Token Probabilities: Providing an API method to query the probability of a specific token would be a great addition. This feature would enable more targeted data analysis and could enhance the usability of the API for various use cases.

Implementing these features could greatly increase the API's flexibility and utility for developers and researchers. I would appreciate any consideration you could give to these suggestions, and I am eager to assist with testing or feedback if needed.

Thank you for your time and all the work you do on this project.

xufengduan avatar Apr 20 '24 11:04 xufengduan