text-generation-inference
text-generation-inference copied to clipboard
Wrong validations on `Parameters` in TGI python library
System Info
NA
Information
- [X] Docker
- [X] The CLI directly
Tasks
- [X] An officially supported command
- [ ] My own modifications
Reproduction
Frequency penalty is validated to be strictly positive in the python library type validation, but is validated to be in the range [-2.0, 2.0] in the actual webserver. This means that i can send a REST API request to TGI with frequency penalty < 0 but will run into validation error when doing it via the text generation client
from text_generation import Client
endpoint_url = "https://YOUR_ENDPOINT.endpoints.huggingface.cloud"
client = Client(endpoint_url)
text = client.generate("Why is the sky blue?", frequency_penalty=-1.0).generated_text
print(text)
# ' Rayleigh scattering'
Expected behavior
No validation errors for frequency_penalty in the range [-2.0, 2.0]