huggingface_hub icon indicating copy to clipboard operation
huggingface_hub copied to clipboard
verified publisher icon huggingface

Support `/info` and `/health` routes in InferenceClient

Open Wauplin opened this issue 1 year ago • 3 comments

Close https://github.com/huggingface/huggingface_hub/issues/1819

cc @MoritzLaurer @thomwolf who requested it

Notes:

  • get_endpoint_info only available on TGI/TEI-powered models
  • health_check only available on TGI/TEI-powered models and only in InferenceEndpoint/local deployment. For serverless InferenceAPi, better to use get_model_status.

Wauplin avatar May 03 '24 13:05 Wauplin

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

HuggingFaceDocBuilderDev avatar May 03 '24 13:05 HuggingFaceDocBuilderDev

related PR merged in TGI: https://github.com/huggingface/text-generation-inference/pull/1854

drbh avatar May 03 '24 14:05 drbh

Thanks for linking and thanks for reviewing/merging the other PR @drbh!

Wauplin avatar May 03 '24 15:05 Wauplin

Thanks for the review!

Wauplin avatar May 27 '24 16:05 Wauplin