google-airflow-premier-league
google-airflow-premier-league copied to clipboard
broken dag -> premier_league_data_to_gc.py missing module util.
Great blog post. But I had consistent broken module with "util" missing.
error on Dataflow
Broken DAG: [/home/airflow/gcs/dags/dags/premier_league_data_to_gc.py] No module named 'util'
After modifying the code as follows, it works fine:
google_init.sh
echo "deploy airflow DAG's" && \
BUCKET_NAME=$(gcloud composer environments describe ${GC_PROJECT_ID} --location ${LOCATION} --format="get(config.dagGcsPrefix)")
BUCKET_NAME=${BUCKET_NAME%dags}
bash google_deploy.sh && \
gsutil -m cp -r ../google_deploy/dags/* ${BUCKET_NAME}
echo "your premium league environment is now ready and its project id is: ${GC_PROJECT_ID}"
...
What do you think @MarcusBaitz ?