text-generation-inference icon indicating copy to clipboard operation
text-generation-inference copied to clipboard

[DOCS] Add Google Cloud TGI integration via dedicated DLCs

Open alvarobartt opened this issue 1 year ago • 6 comments

What does this PR do?

This PR adds Google Cloud within the TGI Documentation https://huggingface.co/docs/text-generation-inference/main/en/reference/api_reference#cloud-providers side-by-side with the Amazon SageMaker integration.

[!WARNING] Before merging I wanted to first clarify whether it would make sense to have a separate section within the TGI documentation for the Cloud Providers; as not sure that the Google Cloud integration fits within the HTTP API Reference page? Maybe you can help clarify a bit better @nbroad1881

Before submitting

  • [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • [ ] Did you read the contributor guideline, Pull Request section?
  • [ ] Was this discussed/approved via a Github issue or the forum? Please add a link to it if that's the case.
  • [ ] Did you make sure to update the documentation with your changes? Here are the documentation guidelines, and here are tips on formatting docstrings.
  • [ ] Did you write any new necessary tests?

Who can review?

@nbroad1881 @pagezyhf @philschmid

alvarobartt avatar Oct 05 '24 16:10 alvarobartt

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

My 2 cents:

In Reference > API Reference > HTTP API Reference > OpenAI Messages API > Cloud Provider, I would just point to https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-vertex-using-openai-library#supported-model-garden-models

What you wrote fits a generic introduction about using TGI on GCP. I think it's valuable to add it to the TGI doc but this is not the right place. It could be in Tutorials or Getting Started with simple pointers to the existing doc in https://huggingface.co/docs/sagemaker/en/index for AWS and https://huggingface.co/docs/google-cloud/en/index for Google Cloud.

pagezyhf avatar Oct 07 '24 09:10 pagezyhf

In Reference > API Reference > HTTP API Reference > OpenAI Messages API > Cloud Provider, I would just point to https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/call-vertex-using-openai-library#supported-model-garden-models

Agree @pagezyhf, also the AWS SageMaker section there is also covering deployment + inference, which may be out of the scope of the section, right?

What you wrote fits a generic introduction about using TGI on GCP

Yes that's right, that's why I also think that we should maybe have a dedicated entry for those with brief introduction plus pointers to the actual documentation pages we host separately

Thanks for the replies @pagezyhf 🤗 WDYT about those @nbroad1881?

alvarobartt avatar Oct 07 '24 09:10 alvarobartt

Looks great to me! Thanks for doing this so quickly. Sorry I responded so slowly.

nbroad1881 avatar Oct 22 '24 09:10 nbroad1881

I wonder if it would be good to have separate sections in the Getting Started bar for AWS and GCP. It is an important feature that seems very hidden.

image

nbroad1881 avatar Oct 22 '24 09:10 nbroad1881

I wonder if it would be good to have separate sections in the Getting Started bar for AWS and GCP. It is an important feature that seems very hidden.

Agree, mentioned above too; I believe that something like "Using TGI on AWS" and "Using TGI on Google Cloud" may work better (?), happy to work on that for both AWS and Google Cloud if we decide to go that path.

alvarobartt avatar Oct 22 '24 10:10 alvarobartt