Airflow Helm Chart 1.11.0 doesn't fresh install Airflow on an On Premises Kubernetes 1.28.2 Cluster

[gbonazzoli]

Official Helm Chart version

1.11.0 (latest released)

Apache Airflow version

2.7.1

Kubernetes Version

1.28.2

Helm Chart configuration

The vanilla command copied from documentation:

helm upgrade --install airflow apache-airflow/airflow --namespace airflow --create-namespace

does not work.

Its output is:

Release` "airflow" does not exist. Installing it now.
Error: failed post-install: 1 error occurred:
	* timed out waiting for the condition

Docker Image customizations

None

What happened

This is the status during helm install:

root@kube-console:~# k get all -n airflow
NAME                                       READY   STATUS     RESTARTS      AGE
pod/airflow-postgresql-0                   0/1     Pending    0             7m50s
pod/airflow-redis-0                        0/1     Pending    0             7m50s
pod/airflow-run-airflow-migrations-dg9bx   1/1     Running    3 (50s ago)   7m50s
pod/airflow-scheduler-77b484b6d5-hfjlz     0/2     Init:0/1   3 (50s ago)   7m50s
pod/airflow-statsd-7d985bcb6f-77jfj        1/1     Running    0             7m50s
pod/airflow-triggerer-0                    0/2     Pending    0             7m50s
pod/airflow-webserver-95c88f8b8-mgdwt      0/1     Init:0/1   3 (53s ago)   7m50s
pod/airflow-worker-0                       0/2     Pending    0             7m50s

NAME                            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/airflow-postgresql      ClusterIP   10.107.125.183   <none>        5432/TCP            7m50s
service/airflow-postgresql-hl   ClusterIP   None             <none>        5432/TCP            7m50s
service/airflow-redis           ClusterIP   10.111.97.240    <none>        6379/TCP            7m50s
service/airflow-statsd          ClusterIP   10.102.66.204    <none>        9125/UDP,9102/TCP   7m50s
service/airflow-triggerer       ClusterIP   None             <none>        8794/TCP            7m50s
service/airflow-webserver       ClusterIP   10.109.208.113   <none>        8080/TCP            7m50s
service/airflow-worker          ClusterIP   None             <none>        8793/TCP            7m50s

NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/airflow-scheduler   0/1     1            0           7m50s
deployment.apps/airflow-statsd      1/1     1            1           7m50s
deployment.apps/airflow-webserver   0/1     1            0           7m50s

NAME                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/airflow-scheduler-77b484b6d5   1         1         0       7m50s
replicaset.apps/airflow-statsd-7d985bcb6f      1         1         1       7m50s
replicaset.apps/airflow-webserver-95c88f8b8    1         1         0       7m50s

NAME                                  READY   AGE
statefulset.apps/airflow-postgresql   0/1     7m50s
statefulset.apps/airflow-redis        0/1     7m50s
statefulset.apps/airflow-triggerer    0/1     7m50s
statefulset.apps/airflow-worker       0/1     7m50s

NAME                                       COMPLETIONS   DURATION   AGE
job.batch/airflow-run-airflow-migrations   0/1           7m50s      7m50s

What you think should happen instead

The last helm chart release 1.10.0 in a matter of seconds was able to install Airflow

How to reproduce

Give the vanilla command, also gave it on a Kubernetes 1.27.1 Cluster, sam results.

Anything else

No

Are you willing to submit PR?

• [ ] Yes I am willing to submit a PR!

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[hussein-awala]

pod/airflow-scheduler-77b484b6d5-hfjlz 0/2 Init:0/1 3 (50s ago)

Did you check why the scheduler (and the other services) pods restart? Do you have any error in the log?

[gbonazzoli]

@hussein-awala

looking at the events collected by the cluster during helm install

kube@kube-console:~$ kubectl get events --sort-by='.metadata.creationTimestamp' -A | cut -c1-210
NAMESPACE   LAST SEEN   TYPE      REASON              OBJECT                                            MESSAGE
airflow     3m33s       Normal    ScalingReplicaSet   deployment/airflow-scheduler                      Scaled up replica set airflow-scheduler-777f7947dc to 1
airflow     3m33s       Warning   FailedScheduling    pod/airflow-redis-0                               0/4 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/4 nodes are avail
airflow     3m33s       Normal    SuccessfulCreate    statefulset/airflow-postgresql                    create Pod airflow-postgresql-0 in StatefulSet airflow-postgresql successful
airflow     3m33s       Normal    Scheduled           pod/airflow-statsd-786d447967-l9w72               Successfully assigned airflow/airflow-statsd-786d447967-l9w72 to kube-node03
airflow     3m33s       Normal    SuccessfulCreate    statefulset/airflow-redis                         create Claim redis-db-airflow-redis-0 Pod airflow-redis-0 in StatefulSet airflow-redis success
airflow     9s          Normal    FailedBinding       persistentvolumeclaim/redis-db-airflow-redis-0    no persistent volumes available for this claim and no storage class is set
airflow     3m33s       Normal    Scheduled           pod/airflow-run-airflow-migrations-thrwm          Successfully assigned airflow/airflow-run-airflow-migrations-thrwm to kube-node02
airflow     101s        Normal    Pulled              pod/airflow-run-airflow-migrations-thrwm          Container image "apache/airflow:2.7.1" already present on machine
airflow     101s        Normal    Created             pod/airflow-run-airflow-migrations-thrwm          Created container run-airflow-migrations
airflow     3m33s       Warning   FailedScheduling    pod/airflow-postgresql-0                          0/4 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/4 nodes are avail
airflow     3m33s       Normal    SuccessfulCreate    job/airflow-run-airflow-migrations                Created pod: airflow-run-airflow-migrations-thrwm
airflow     3m33s       Normal    Scheduled           pod/airflow-scheduler-777f7947dc-4l8bl            Successfully assigned airflow/airflow-scheduler-777f7947dc-4l8bl to kube-node03
airflow     104s        Normal    Pulled              pod/airflow-scheduler-777f7947dc-4l8bl            Container image "apache/airflow:2.7.1" already present on machine
airflow     104s        Normal    Created             pod/airflow-scheduler-777f7947dc-4l8bl            Created container wait-for-airflow-migrations
airflow     104s        Normal    Started             pod/airflow-scheduler-777f7947dc-4l8bl            Started container wait-for-airflow-migrations
airflow     3m33s       Normal    SuccessfulCreate    replicaset/airflow-scheduler-777f7947dc           Created pod: airflow-scheduler-777f7947dc-4l8bl
airflow     3m33s       Normal    SuccessfulCreate    statefulset/airflow-worker                        create Claim logs-airflow-worker-0 Pod airflow-worker-0 in StatefulSet airflow-worker success
airflow     3m33s       Normal    SuccessfulCreate    statefulset/airflow-postgresql                    create Claim data-airflow-postgresql-0 Pod airflow-postgresql-0 in StatefulSet airflow-postgresql success
airflow     3m33s       Normal    SuccessfulCreate    statefulset/airflow-redis                         create Pod airflow-redis-0 in StatefulSet airflow-redis successful
airflow     9s          Normal    FailedBinding       persistentvolumeclaim/logs-airflow-worker-0       no persistent volumes available for this claim and no storage class is set
airflow     9s          Normal    FailedBinding       persistentvolumeclaim/logs-airflow-triggerer-0    no persistent volumes available for this claim and no storage class is set
airflow     3m33s       Normal    SuccessfulCreate    replicaset/airflow-statsd-786d447967              Created pod: airflow-statsd-786d447967-l9w72
airflow     3m33s       Normal    ScalingReplicaSet   deployment/airflow-statsd                         Scaled up replica set airflow-statsd-786d447967 to 1
airflow     3m33s       Warning   FailedScheduling    pod/airflow-triggerer-0                           0/4 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/4 nodes are avail
airflow     3m33s       Normal    SuccessfulCreate    statefulset/airflow-triggerer                     create Claim logs-airflow-triggerer-0 Pod airflow-triggerer-0 in StatefulSet airflow-triggerer success
airflow     3m33s       Normal    SuccessfulCreate    statefulset/airflow-triggerer                     create Pod airflow-triggerer-0 in StatefulSet airflow-triggerer successful
airflow     3m33s       Normal    Scheduled           pod/airflow-webserver-54644b47bf-pd9mf            Successfully assigned airflow/airflow-webserver-54644b47bf-pd9mf to kube-node03
airflow     103s        Normal    Pulled              pod/airflow-webserver-54644b47bf-pd9mf            Container image "apache/airflow:2.7.1" already present on machine
airflow     103s        Normal    Created             pod/airflow-webserver-54644b47bf-pd9mf            Created container wait-for-airflow-migrations
airflow     103s        Normal    Started             pod/airflow-webserver-54644b47bf-pd9mf            Started container wait-for-airflow-migrations
airflow     3m33s       Normal    SuccessfulCreate    replicaset/airflow-webserver-54644b47bf           Created pod: airflow-webserver-54644b47bf-pd9mf
airflow     3m33s       Normal    ScalingReplicaSet   deployment/airflow-webserver                      Scaled up replica set airflow-webserver-54644b47bf to 1
airflow     3m33s       Warning   FailedScheduling    pod/airflow-worker-0                              0/4 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/4 nodes are avail
airflow     9s          Normal    FailedBinding       persistentvolumeclaim/data-airflow-postgresql-0   no persistent volumes available for this claim and no storage class is set
airflow     3m33s       Normal    SuccessfulCreate    statefulset/airflow-worker                        create Pod airflow-worker-0 in StatefulSet airflow-worker successful
airflow     101s        Normal    Started             pod/airflow-run-airflow-migrations-thrwm          Started container run-airflow-migrations
airflow     3m32s       Normal    Started             pod/airflow-statsd-786d447967-l9w72               Started container statsd
airflow     3m32s       Normal    Created             pod/airflow-statsd-786d447967-l9w72               Created container statsd
airflow     3m32s       Normal    Pulled              pod/airflow-statsd-786d447967-l9w72               Container image "quay.io/prometheus/statsd-exporter:v0.22.8" already present on machine

I think that the problem is caused by:

pod has unbound immediate PersistentVolumeClaims. preemption: 0/4 nodes are avail

Actuallly the cluster does not have anything related to dinamically create PV from a block storage.

It is now a requirement ?

[romsharon98]

It's look like all the containers that use persistent volume failed. Did you check all the persistent volumes was created successfully?

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.

[github-actions[bot]]

This issue has been closed because it has not received response from the issue author.