CTranslate2
CTranslate2 copied to clipboard
Output tokens logits
Hello,
Unless I'm mistaken, I don't see any option in the translator translate_batch or generate_tokens functions to output the logits/probabilities of the generated tokens. However, this computation must be done internally before sampling the tokens, right ?
This would be very useful in order to have more control over the generation, for example for influencing the choice of specific tokens. Is there a particular reason why this is not exposed ? Could this be easily added, or am I missing something? This could for example take the form of a return_logits argument to the mentioned functions.
Thanks, Simon