Huggingface Agents Error 422: {'error': 'Input validation error: `max_new_tokens` must be <= 192

[dashapetr]

System Info

Transformers v4.29.0, v4.36.2

Who can help?

@ArthurZucker, @younesbelkada

Information

• [X] The official example scripts
• [X] My own modified scripts

Tasks

• [X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [ ] My own task or dataset (give details below)

Reproduction

Run in Google Colab:

#@title Setup
transformers_version = "v4.36.2"  # you can use "v4.29.0", the issue and output are the same

print(f"Setting up everything with transformers version {transformers_version}")

!pip install huggingface_hub>=0.14.1 git+https://github.com/huggingface/transformers@$transformers_version -q diffusers accelerate datasets torch soundfile sentencepiece opencv-python openai

from huggingface_hub import notebook_login
notebook_login()

from transformers import HfAgent

agent = HfAgent("https://api-inference.huggingface.co/models/bigcode/starcoder", token='hf_my_token')  # token is passed directly here to avoid the issue https://github.com/huggingface/transformers/issues/28217
agent.run("Is the following `text` (in Spanish) positive or negative?", text="¡Este es un API muy agradable!")

Expected behavior

It should generate results, but instead, I am getting an error:

ValueError: Error 422: {'error': 'Input validation error: max_new_tokens must be <= 192. Given: 200', 'error_type': 'validation'}

To my mind, it seems like it could be related to a strict limitation on max_new_tokens here

[ArthurZucker]

Hey! Thanks, you are probably right. Would you like to open a PR to change this and make it more friendly?

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.