Reduce memory allocation when diff'ing large manifests

[sjentzsch]

With latest kapp-controller v0.48.1 (and related kapp version) we face the following issue:

We have an app with large manifests (based on https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack), namely about 4.4MB of manifest files. When we release a new version of this app, and say all labels do change, so the diff is large, kapp seems to allocate so much memory, that it crashed on our system with 4GB of memory allocated for the container (Reconcile failed: Deploying: signal: killed). After providing about 7GB of memory (limits), it finally could reconcile successfully.

This issue is about tracking this issue, and improving upon the memory allocation required for large diffs.

As @praveenrewar wrote:

The total memory allocation is almost 3 times when there is diff vs when there is no diff. I will try to look into it, meanwhile I think you should be able to reduce the memory limits for the container, unless you are planning to bump the helm chart version again. Also, feel free to create an issue on GitHub for tracking the progress.

Related Slack channel: https://kubernetes.slack.com/archives/CH8KCCKA5/p1686575635485759

(maybe also relates to https://github.com/carvel-dev/kapp/issues/599 ?)