airflow-operator icon indicating copy to clipboard operation
airflow-operator copied to clipboard

Allow dags_in_image option for k8s

Open cal-pratt opened this issue 5 years ago • 0 comments

I was trying to run this project using k8s executor and the dags_in_image option and having no dags section

apiVersion: airflow.k8s.io/v1alpha1
kind: AirflowCluster
metadata:
  name: pk-cluster
spec:
  executor: Kubernetes
  config:
    airflow:
      AIRFLOW__KUBERNETES__DAGS_IN_IMAGE: "1"
  ui:
    image: "my-custom-airflow-image"
    version: "1.10.2"
    replicas: 1
  scheduler:
    image: "my-custom-airflow-image"
    version: "1.10.2"
  worker:
    image: "my-custom-airflow-image"
    version: "1.10.2"
  airflowbase:
    name: pc-base

However, I was getting errors when trying to run the operator:

  Conditions:
    Last Transition Time:  2019-06-06T06:39:42Z
    Last Update Time:      2019-06-06T06:39:42Z
    Message:               templates/airflow-configmap.yaml:template: tmpl:174:25: executing "tmpl" at <.Cluster.Spec.DAGs.G...>: can't evaluate field Repo in type *v1alpha1.GitSpec
    Reason:                ErrorSeen
    Status:                True
    Type:                  Error

I believe the issue is caused by doing a hardcoded lookup on the spec values for git https://github.com/GoogleCloudPlatform/airflow-operator/blob/980a1848857ed86eb44a4e61c4c214207dad67be/templates/airflow-configmap.yaml#L174-L180

Not exactly sure how to fix this in a proper manner, but removing these lines in the operator image was enough to get me past this issue.

Thanks a lot

cal-pratt avatar Jun 07 '19 16:06 cal-pratt