Karl Isenberg
Karl Isenberg
For comparison, here's the same number of deployments but more reconcilers (20x100). Interestingly, the peak memory use is basically the same as the 10x200 test, even with half the objects...
Here's 20x200 (4000 deployments total). With this configuration, some of the reconcilers are reaching the new lower 436Mi memory limit, but it still completed. From this we can clearly see...
At 30x200 the test fails to complete. The reconcilers are repeatedly OOMKilled before they can finish syncing. It seems like maybe autoscaling based purely on the number of objects being...
So I figured out how to modify the remediator to filter watches by label and added a filter that should only watch resources with the correct `cli-utils.sigs.k8s.io/inventory-id` label for that...
Turns out, the label selector I used for the previous test was filtering out all objects, but the test still passed because remediation wasn't required for the test to pass....
10x500 seems to cap out a little higher at ~170Mi ![Screenshot from 2024-04-03 18-09-51](https://github.com/GoogleContainerTools/kpt-config-sync/assets/3980984/18c48f53-9a62-4553-87eb-575571d3d129)
10x700 got up to ~200Mi ![Screenshot from 2024-04-03 18-43-07](https://github.com/GoogleContainerTools/kpt-config-sync/assets/3980984/737db681-3c1c-4410-9ba4-b530f8ef2184)
Having quota problems with larger numbers of nodes, so here's 3x990. Keeping it just under the 1000 count, but it obviously doesn't need even half of the limit here. Maxing...
2x2000 here. Maxes out at ~275Mi ![Screenshot from 2024-04-03 20-29-02](https://github.com/GoogleContainerTools/kpt-config-sync/assets/3980984/d058e966-7844-4278-8ac2-e6741cac4cfc)
1x5000 here. Maxes out at ~525Mi ![Screenshot from 2024-04-03 20-59-20](https://github.com/GoogleContainerTools/kpt-config-sync/assets/3980984/e268cccd-f8be-4a20-9890-462187df9a9e)