kubernetes
kubernetes copied to clipboard
Minio pod crashes after updating from 24.1.0 to 24.1.1
After updating with existing data from 24.1.0
to 24.1.1
the minio container crashes.
We tested the minio without providing data and it works.
In 24.1.1
minio has been updated - from 2021.4.22-debian-10-r6
to 2024.5.10-debian-12-r2
(link to diff)
Error overview:
- The minio container was first crashing because it did not meet its
readinessProbe
constraints - After disabling both readiness and liveness probes the container kept crashing - with the following error:
Defaulted container "minio" out of: minio, volume-permissions (init)
10:58:49.36 INFO ==>
10:58:49.36 INFO ==> Welcome to the Bitnami minio container
10:58:49.36 INFO ==> Subscribe to project updates by watching https://github.com/bitnami/containers
10:58:49.36 INFO ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
10:58:49.36 INFO ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
10:58:49.36 INFO ==>
10:58:49.36 INFO ==> ** Starting MinIO setup **
minio 10:58:49.38 DEBUG ==> Validating settings in MINIO_* env vars..
minio 10:58:49.40 INFO ==> Starting MinIO in background...
ERROR Unable to use the drive /data: drive not found: Invalid arguments specified
minio 10:58:59.50 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:04.58 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:09.68 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:14.77 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:19.88 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:24.98 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:30.07 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:35.18 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:40.28 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:45.38 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:50.48 INFO ==> Adding local Minio host to 'mc' configuration...
minio 10:59:55.58 INFO ==> Adding local Minio host to 'mc' configuration...
Failed to add temporary MinIO server
minio 11:00:00.68 INFO ==> MinIO is already stopped...
This is our minio configuration in values.yaml
(updated from the old-minio accesskey
/secretkey
to rootUser
/rootPassword
):
minio:
image:
debug: true
auth:
rootUser: "miniouser"
rootPassword: "miniopassword"
persistence:
existingClaim: minio-data
mountPath: /data
volumePermissions:
enabled: true
livenessProbe:
enabled: false
readinessProbe:
enabled: false
About ERROR Unable to use the drive /data: drive not found: Invalid arguments specified
- the /data
dir is set to the correct minio container user (1001
) due to volumePermissions.enabled=true
setting:
I have no name!@my-release-minio-6b9666d756-qw9hj:/$ ls -la / | grep data
drwxrwsr-x 81 1001 1001 4096 Jul 9 08:42 data
And all data is indeed there:
I have no name!@my-release-minio-7b4bdb744d-q79b7:/data$ ls -la
total 356
drwxrwsr-x 81 1001 1001 4096 Jul 9 08:42 .
drwxr-xr-x 3 root root 18 Jun 26 05:01 ..
-rw------- 1 1001 1001 6 Jul 9 11:49 .access_key
drwxrwsr-x. 7 1001 1001 4096 Jul 9 11:57 .minio.sys
-rw------- 1 1001 1001 9 Jul 9 11:57 .root_password
-rw------- 1 1001 1001 6 Jul 9 11:57 .root_user
-rw------- 1 1001 1001 12 Jul 9 11:49 .secret_key
drwxrws--- 2 1001 1001 16384 Feb 29 15:39 lost+found
drwxrwsr-x. 2 1001 1001 4096 Feb 29 17:47 prj-1
drwxrwsr-x. 4 1001 1001 4096 Feb 29 17:47 prj-102
.................................
Reverting back to 24.1.0
-> all is working correctly with the data.
Any idea?