private-llm-aws icon indicating copy to clipboard operation
private-llm-aws copied to clipboard

Published 20 hours ago • verified publisher icon singlestore-labs

TF Deploy FastAPI service on AppRunner

Open wesdottoday opened this issue 2 years ago • 0 comments

  • [ ] Take ECR URI and create an AppRunner service with appropriate environment variables
  • [ ] Store AppRunner URI in TF Variable
  • [ ] Store FastAPI API Key (generated on initial boot) and store in TF
  • [ ] Liveness test, communicate with SageMaker LLM and Embedding model
  • [ ] Provide connection_string for SingleStoreDB
  • [ ] Liveness test, communicate with S2DB

The initial deployment of the FastAPI service relies on the underlying issues #3 and #4 to be functioning and deployed. This service will be what communicates back and forth with SageMaker.

wesdottoday avatar Sep 26 '23 19:09 wesdottoday