Simpler deployment

[tstadel]

What?

Guide / template to deploy on AWS / Kubernetes

Why?

Road to production from an experimental environment should be simplified.

How?

• create a simple helm chart for AWS EKS that anybody can deploy via AWS console
• this helm start works as a starting point to make more complex deployments happen (e.g. scaling, different k8s flavour, different ingress)
• find a way to deliver helm chart (registry is just a http server)
• write a guide in our documentation how to use it

To be refined

• MVP is AWS/Kubernetes deployment? Yes, we use Helm Charts. There are various advantages compared to docker-compose like k8s' scaling, resilience and deployment capabilities. Additionally, K8s has become defacto-standard for deployment.
• Is ray too complex here? Yes, might be another Epic.
• Do we want to support other environments (like Azure, Google Cloud)? Yes, Helm Chart should be compatible with other Kubernetes flavours
• Are there more things to be considered if pipelines get more complex than say a basic Retriever-Reader-Pipeline?
  • GPU allocation
• Can we generate Helm charts from docker-compose.yaml? No, helm is more specific. Helm charts created by kompose are too basic (e.g. pods instead of deployments, no ingress)

Misc

• Convert docker-compose.yaml to helm chart: https://kompose.io/installation/#ubuntudebian
• Deploy docker-compose.yaml on AWS: https://aws.amazon.com/blogs/containers/deploy-applications-on-amazon-ecs-using-docker-compose/
• Deploy docker-compose.yaml on Azure: https://docs.microsoft.com/en-us/azure/container-instances/tutorial-docker-compose
• Deploy helm chart on AWS: https://docs.aws.amazon.com/eks/latest/userguide/helm.html https://docs.aws.amazon.com/AmazonECR/latest/userguide/push-oci-artifact.html

[mumtazcem]

Hello,

I have been deploying my simple fastapi app which includes haystack API to aws EC2 instance. So far I was able to run PDFToTextConverter, PreProcessor, ElasticsearchDocumentStore, FARMReader without any problem. Yet Crawler is not working for some reason, I suspect that it might be related to the chromedriver. It does not raise any errors. It gets the url, creates the json file under crawled_files folder with empty content.

Chromedriver installation is problematic and it is mentioned in this post : https://forums.aws.amazon.com/thread.jspa?threadID=299727.

I downloaded it via this post though it is not official https://understandingdata.com/install-google-chrome-selenium-ec2-aws/.

I did not use Kubernetes though, I don't know its capabilities, but chromedriver may cause some issues.

[MichelBartels]

Hi @mumtazcem. Thanks for the input. We will definitely be considering this when working on the issue. However, this issue is currently in our backlog so please don't expect any progress too soon.

[mumtazcem]

Thanks for the insight @MichelBartels. I am following this issue, and I will be glad to help, please let me know.

[mumtazcem]

Hello, Quick update. I managed to deploy a simple Retriever-Reader-Pipeline fastapi application with docker-compose to AWS. I have used elasticsearch and selenium images and made some changes in Crawler. It consumes around 3gb memory.

Given urls and questions list, it returns question and answers.

If this is the way to go for 'Simpler deployment', I can work on PR to convert my docker-compose.yaml to helm chart, and make necessary changes.

Example:

{ "url_list": ["https://www.adaptiv-networks.com/post/qsrs-and-fast-casual-restaurants-are-improving-operations-with-adaptiv-sd-wan"], "questions_list": ["What is the challenge?"] }

{ "Question": "What is the challenge?", "Answers": { "answer": "restaurant teams struggle to succeed when their network connectivity is not delivering", "score": 0.22019997239112854, "context": "onnectivity for staff training, restaurant teams struggle to succeed when their network connectivity is not delivering. Adaptiv Eliminates the Challen", "document_id": "34ab9a8ae484e4bfe5c27ef6c814bef6", "meta": { "url": "https://www.adaptiv-networks.com/post/qsrs-and-fast-casual-restaurants-are-improving-operations-with-adaptiv-sd-wan", "_split_id": 1 } } }

[tstadel]

Hey @mumtazcem, good to here that you managed to deploy your pipeline :-) We already started on creating a simple helm chart, but it's still a draft. It works on a local cluster, but we didn't try to use it on a public cloud provider (e.g. AWS EKS). Checkout https://github.com/deepset-ai/haystack/pull/2295. We planned to make progress on this by next week. Next steps will be to make it work with AWS EKS. However, this PR focuses solely on a Retriever-Reader-Pipeline without a crawler and that should stay its focus as a first minimal version.

If you like to, however, you can create a PR on top of this that includes the crawler and try to make it work with AWS EKS. That way we should end up with generic changes that we can merge back into the PR and a "Crawler Add-on" to that PR which we can deliver in the future. What do you think?

[mumtazcem]

That's awesome. I would love to work on "Crawler Add-on". I will keep you updated.

[masci]

The scope of the epic is too broad and Haystack's deployment is becoming a hot topic in the Community - I'm going to close this epic as done limited to the Helm chart work, following up on the remaining topics with different streams of work that will be tracked in different epics/tasks.