data-pipelines-with-apache-airflow
data-pipelines-with-apache-airflow copied to clipboard
BasPH
Code for Data Pipelines with Apache Airflow
Running the tutorial on Digital Ocean Droplet had to make the following edits to the code to get it to run: 1) Update Import Statements to : `from airflow.operators.bash_operator import...
While cloning the repo, git complains about 'colliding group is in the working tree'. There are two 'readme' files (one all upper case and one in lowercase) with identical contents....
I tried to run figure_6_20.py. ExternalTaskSensor in dag2 remains in running state even the `process_market` task in dag1 is success. Could anyone give me a pointer on this issue? Thanks.
when you use "curl -o /data/events.json http://events_api:5000/events" and say "fetch and store the events from the API" What is that API? I assume this DAG won't run? not sure if...
Hi team, the request is to add best practice(s) around mocking metastore variables for use in unit tests. I've been able to pull together the following code in my code...
the code license is missing. Is it yet defined and can the code presented here be re-used, resp. under which conditions?
The 11_xcoms_return.py should modify to use "**return_value**" as XComs pull key as indicated in Figure 5.17. From: ```python deploy_model = PythonOperator( task_id="deploy_model", python_callable=_deploy_model, templates_dict={ "model_id": "{{task_instance.xcom_pull(task_ids='train_model', key='model_id')}}" }, ) ```...
The dag id is named as `listing_6_08` in `chapter06/dags/listing_6_8.py`, so the `chapter06/scripts/trigger_dag.sh` should follow the dag id. From: ```bash #!/usr/bin/env bash # Trigger DAG with Airflow CLI airflow dags trigger...