Max token input window & completion output

[ganymedenet]

Good day,

Model: 7B

• What is the maximum token input window?
• How can I set or limit the number of completion tokens?

No matter how many prompt_tokens I supply, I always receive 16 completion tokens back.

   "usage": {
      "prompt_tokens": 191,
      "completion_tokens": 16,
      "total_tokens": 207
   }

[Anoerak]

Based on what I've tried, at least with my server in a Docker container, the maximum token I can get is 100. In order to change the total_tokens, you need to add a "max_token" parameter to your request.

https://platform.openai.com/docs/api-reference/chat/create#max_tokens

[ganymedenet]

Thank you!

Based on what I've tried, at least with my server in a Docker container, the maximum token I can get is 100. In order to change the total_tokens, you need to add a "max_token" parameter to your request.

https://platform.openai.com/docs/api-reference/chat/create#max_tokens

Thank you! It works..

[Anoerak]

Nice :) You can close the issue I guess.