Provide logprobs in the API.

[BenjaminMarechalEVITECH]

Hi,

Is there any way to get probabilities of tokens from moondream models ? In OpenAI API, this is called "logprobs" (https://platform.openai.com/docs/api-reference/chat/create) If not, that would be a be a great add-on. For example, it is useful for image classification (query with "yes/no" answer).

Benjamin

[vikhyat]

Not possible right now, but we'll look into adding this! Will leave this open in the mean time.

[anerli]

+1, especially for the point API in particular. For query as well would be great. Would be extremely useful to us. Curious what makes it technically difficult?

[vikhyat]

We're building a new version of our cloud inference stack, it will be in the next version! Not super technically hard, but I didn't want to spend the time reimplementing it in the old stack.