Kubernetes Executor: worker pods are not executing

[jcunhasilva]

I was testing the fixes made here: https://github.com/GoogleCloudPlatform/airflow-operator/issues/31

Although Git authentication is working now, I am still not able to run our workers using the Kubernetes executor.

When the worker pod is launched no work is done, or at least the scheduler doesn't get any feedback and the UI doesn't display any execution updates regarding the DAG run.

[barney-s]

Since i dont know what DAGs you are running can you try this ?

Set these to false: https://github.com/GoogleCloudPlatform/airflow-operator/blob/master/templates/airflow-configmap.yaml#L172 https://github.com/GoogleCloudPlatform/airflow-operator/blob/master/pkg/apis/airflow/v1alpha1/airflow.go#L194

Try running again and look at the logs from the worker pods. You would need to list completed pods too.

[jcunhasilva]

@barney-s we still have the git authentication issue initialising the worker pods. Git credentials are not available:

  initContainers:
  - env:
    - name: GIT_SYNC_REPO
      value: https://github.com/MyOrg/my-repo.git
    - name: GIT_SYNC_BRANCH
      value: development
    - name: GIT_SYNC_ROOT
      value: /git
    - name: GIT_SYNC_DEST
      value: gitdags
    - name: GIT_SYNC_DEPTH
      value: "1"
    - name: GIT_SYNC_ONE_TIME
      value: "true"
    image: gcr.io/google-containers/git-sync-amd64:v2.0.5

[barney-s]

Sent a patch. Could you please test if it works ?

[barney-s]

Potential fix here: https://github.com/barney-s/airflow-operator/commit/5bc91cc37f01cdc4367de4984418e2b63ca49c80

Would you be able to verify/use this ?