microservices-demo
microservices-demo copied to clipboard
Base resource requests on actual usage
Background
- This fixes https://github.com/GoogleCloudPlatform/microservices-demo/issues/591.
- I looked at a deployment of Online Boutique from 4 days ago. I noted the max (actually used) vCPU and memory values of each microservice — within the full 4 days. (The deployment included the loadgenerator.) Olivier did a similar analysis in 2022.
- For each value, I rounded up to the next
10m(for CPU) and the next10MiBfor memory — to add a buffer.
Change Summary
- I've updated the
resource.requests.cpuandresource.requests.memoryvalues of each default microservices's Deployment.
Testing Procedure
- We just have to make sure the staging URL works fine. :)
🚲 PR staged at http://34.42.127.234
🚲 PR staged at http://34.42.127.234
I have tested the new Kubernetes manifests on the regular channel and rapid channel (see delete-me-test-resources-new and delete-me-test-resources-new-rapid below).
The new manifests work.
However, I believe, on GKE autopilot, this only shaved off 0.25 vCPUs because the Managed Service for Prometheus is enabled by default on GKE autopilot and requires a lot (relative to Online Boutique) of resources.
🚲 PR staged at http://34.42.127.234
🚲 PR staged at http://34.42.127.234
FYI: I just deployed this in my own GKE Autopilot cluster, and I'm getting these values, k top pods -n onlineboutique-development:
NAME CPU(cores) MEMORY(bytes)
adservice-bc9667f7c-cdbrd 254m 128Mi
cartservice-69754cfcc9-pvf88 7m 89Mi
checkoutservice-856c5f7f68-cwnnn 84m 49Mi
currencyservice-66cc88ff55-5mk9l 11m 74Mi
emailservice-7d69bb6774-cvx2b 14m 81Mi
frontend-7fcf78c9bd-nxrz8 4m 50Mi
loadgenerator-76f787d859-nthg6 12m 142Mi
paymentservice-7495f4f8d9-b94rh 4m 66Mi
productcatalogservice-5dffd678c-8b7nn 5m 51Mi
recommendationservice-ff8fcbf9d-cgbjz 6m 91Mi
redis-575569978c-2z5fh 6m 40Mi
shippingservice-7b6764896f-9zhd5 5m 47Mi
In addition to the comment on the recommendationservice, I'm getting these restarts because of memory limits too restrictive:
adservice-bc9667f7c-cdbrd 1/2 CrashLoopBackOff 115 (72s ago) 5h39m
checkoutservice-856c5f7f68-cwnnn 2/2 Running 1 (7m50s ago) 5h39m
currencyservice-66cc88ff55-5mk9l 2/2 Running 5 (62m ago) 5h39m```
Should we have:
- For
adservice:150Miformemorylimits? and150mforcpulimits? - For
checkoutservice:80Miformemorylimits? - For
currencyservice:90Miformemorylimits?
🚲 PR staged at http://34.42.127.234
@NimJay do you remember where this PR was left off?