m_params.max_tokens set to small number will return cut off text

[RndyP]

Environment: I am sending Whisper short chunks of about 3 seconds of audio. I set the parameter m_params.max_tokens=3 because I am not expecting many tokens in this amount of time.

Problem: The returned text from whisper_full_get_segment_text is returned chopped off text in the case where whisper is returning what I will call "enviroment" text. This is like:

*wind blowing" What do you call these?? Anyway...

The return string is missing the '*' or ']' Will return:

*wind blowing

If I set m_params.max_tokens=0, it is not chopped.

If I set m_params.max_tokens=4, it is not chopped.

Perhaps *wind blowing requires 4 tokens?

[ggerganov]

Yes, I think the tokens are as follows:

• token 0: *
• token 1: wind
• token 2: blowing