text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

Wrong validations on `Parameters` in TGI python library

Open Jason-CKY opened this issue 1 year ago • 0 comments

System Info

NA

Information

  • [X] Docker
  • [X] The CLI directly

Tasks

  • [X] An officially supported command
  • [ ] My own modifications

Reproduction

Frequency penalty is validated to be strictly positive in the python library type validation, but is validated to be in the range [-2.0, 2.0] in the actual webserver. This means that i can send a REST API request to TGI with frequency penalty < 0 but will run into validation error when doing it via the text generation client

from text_generation import Client

endpoint_url = "https://YOUR_ENDPOINT.endpoints.huggingface.cloud"

client = Client(endpoint_url)
text = client.generate("Why is the sky blue?", frequency_penalty=-1.0).generated_text
print(text)
# ' Rayleigh scattering'

Expected behavior

No validation errors for frequency_penalty in the range [-2.0, 2.0]

Jason-CKY avatar May 17 '24 02:05 Jason-CKY