datahub-study-curation-tools icon indicating copy to clipboard operation
datahub-study-curation-tools copied to clipboard

Genie import DAG prototype

Open callachennault opened this issue 1 year ago • 0 comments

This PR introduces two DAGs: genie_data_dag.py, and import_genie_dag.py.

  • Scripts for the genie_data_dag.py are contained in the scripts/ directory. Also included in this PR is a Dockerfile for the Airflow worker pods, along with an executor config definition and a python requirements file.

  • Scripts for the import_genie_dag.py are contained in the cmo-pipelines repository: https://github.com/knowledgesystems/cmo-pipelines/pull/1201

  • [ ] DISCUSS: Should the import dag scripts be moved to this repo?

callachennault avatar Nov 27 '24 21:11 callachennault