datahub-study-curation-tools
datahub-study-curation-tools copied to clipboard
Genie import DAG prototype
This PR introduces two DAGs: genie_data_dag.py, and import_genie_dag.py.
-
Scripts for the genie_data_dag.py are contained in the
scripts/directory. Also included in this PR is a Dockerfile for the Airflow worker pods, along with an executor config definition and a python requirements file. -
Scripts for the
import_genie_dag.pyare contained in the cmo-pipelines repository: https://github.com/knowledgesystems/cmo-pipelines/pull/1201 -
[ ] DISCUSS: Should the import dag scripts be moved to this repo?