google-airflow-premier-league icon indicating copy to clipboard operation
google-airflow-premier-league copied to clipboard

broken dag -> premier_league_data_to_gc.py missing module util.

Open djoguns opened this issue 4 years ago • 0 comments

Great blog post. But I had consistent broken module with "util" missing.

error on Dataflow

Broken DAG: [/home/airflow/gcs/dags/dags/premier_league_data_to_gc.py] No module named 'util'

After modifying the code as follows, it works fine:

google_init.sh

echo "deploy airflow DAG's" && \
BUCKET_NAME=$(gcloud composer environments describe ${GC_PROJECT_ID} --location ${LOCATION} --format="get(config.dagGcsPrefix)")
BUCKET_NAME=${BUCKET_NAME%dags}

bash google_deploy.sh && \

gsutil -m cp -r ../google_deploy/dags/* ${BUCKET_NAME}

echo "your premium league environment is now ready and its project id is: ${GC_PROJECT_ID}"
...

What do you think @MarcusBaitz ?

djoguns avatar Mar 18 '20 13:03 djoguns