charts icon indicating copy to clipboard operation
charts copied to clipboard

bitnami/mongodb: Fix zombie processes by probes

Open pmhahn opened this issue 1 year ago • 2 comments

Description of the change

Fix mongdb chart to not use shell constructs in startup / readiness / livenes probes which may leave behind zombie processes.

Benefits

Running mongoDB for a long time (as part of Graylog) may leave behind 1000+ zombie processes which will eventually starve the host.

Possible drawbacks

No.

Applicable issues

  • fixes bitnami/charts#13365

Additional information

Not yet tested. Please review carefully.

Checklist

  • [x] Chart version bumped in Chart.yaml according to semver. This is not necessary when the changes only affect README.md files.
  • [ ] Variables are documented in the values.yaml and added to the README.md using readme-generator-for-helm
  • [x] Title of the pull request follows this pattern [bitnami/<name_of_the_chart>] Descriptive title
  • [x] All commits signed off and in agreement of Developer Certificate of Origin (DCO)

pmhahn avatar Feb 11 '24 14:02 pmhahn

@javsalgar can you please assist us with this pull request?

gituser8796 avatar Feb 11 '24 16:02 gituser8796

  • [x] I tested this (94a441951316f9968693b726c16b471b6082429b) now on my private setup by doing the equivalent of kill -STOP 1 on mongod; after some time k8s detected the failure and restarted the process:
# helm install -n phahn-mongodb --create-namespace mongodb ./bitnami/mongodb
# kubectl -n phahn-mongodb exec mongodb-b57576544-xwdpd -- kill -STOP 1
# kubectl -n phahn-mongodb get events -w
LAST SEEN   TYPE     REASON                  OBJECT                          MESSAGE
61s         Normal   Scheduled               pod/mongodb-b57576544-xwdpd     Successfully assigned phahn-mongodb/mongodb-b57576544-xwdpd to ibm1
59s         Normal   Pulled                  pod/mongodb-b57576544-xwdpd     Container image "docker.io/bitnami/mongodb:7.0.5-debian-11-r6" already present on machine
59s         Normal   Created                 pod/mongodb-b57576544-xwdpd     Created container mongodb
59s         Normal   Started                 pod/mongodb-b57576544-xwdpd     Started container mongodb
69s         Normal   SuccessfulCreate        replicaset/mongodb-b57576544    Created pod: mongodb-b57576544-xwdpd
70s         Normal   WaitForFirstConsumer    persistentvolumeclaim/mongodb   waiting for first consumer to be created before binding
69s         Normal   ScalingReplicaSet       deployment/mongodb              Scaled up replica set mongodb-b57576544 to 1
69s         Normal   ExternalProvisioning    persistentvolumeclaim/mongodb   waiting for a volume to be created, either by external provisioner "rancher.io/local-path" or manually created by system administrator
69s         Normal   Provisioning            persistentvolumeclaim/mongodb   External provisioner is provisioning volume for claim "phahn-mongodb/mongodb"
62s         Normal   ProvisioningSucceeded   persistentvolumeclaim/mongodb   Successfully provisioned volume pvc-96598210-d1ab-4457-b76a-4e40b6909d41
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     Readiness probe failed: Current Mongosh Log ID:   65ca2ae3c1e71507790f0f8e...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     Liveness probe failed: Current Mongosh Log ID:    65ca2aed948c12d07331c215...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     Readiness probe failed: Current Mongosh Log ID:   65ca2aed3787aef21b196ca7...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     Readiness probe failed: Current Mongosh Log ID:   65ca2af71fd95e5b49d7ee21...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     Liveness probe failed: Current Mongosh Log ID:    65ca2b01a398f675fcc29a5a...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     Readiness probe failed: Current Mongosh Log ID:   65ca2b01759b163b44fefe67...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     Readiness probe failed: Current Mongosh Log ID:   65ca2b0be798a2a6c149d311...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     Liveness probe failed: Current Mongosh Log ID:    65ca2b15cf836a1c06efc7be...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     Readiness probe failed: Current Mongosh Log ID:   65ca2b157e1f8bd3a079ea38...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b182967545d7feb6526...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b1f11b6f1b2d00b7241...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Liveness probe failed: Current Mongosh Log ID:    65ca2b2949d0e113cc7302af...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b2934937a5cf7774dce...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b332be5ba2e4cb90645...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Liveness probe failed: Current Mongosh Log ID:    65ca2b3d33a282369ae1e8f6...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b3d25181f3e44a30613...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b477afd1e63cd0d9b93...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Liveness probe failed: Current Mongosh Log ID:    65ca2b51c5e0920f111a59bb...
0s          Normal    Killing                 pod/mongodb-b57576544-xwdpd     Container mongodb failed liveness probe, will be restarted
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b51dadfe1ffb537ae13...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b54a5a0bda3cee992f0...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b5bb42afabe41ebe510...
0s          Warning   Unhealthy               pod/mongodb-b57576544-xwdpd     (combined from similar events): Readiness probe failed: Current Mongosh Log ID:   65ca2b65f212ac966bc60f7a...

pmhahn avatar Feb 12 '24 14:02 pmhahn