cookiecutter-data-science icon indicating copy to clipboard operation
cookiecutter-data-science copied to clipboard

Published 20 hours ago • verified publisher icon drivendataorg

Add option to use different DAG manager

Open pjbull opened this issue 9 years ago • 3 comments

We opt for a Makefile and GNU make by default. Could give options for some of these other projects:

Paver Luigi Airflow Snakemake Ruffus Joblib

pjbull avatar Apr 23 '16 18:04 pjbull

I'm starting to lean Snakemake as the default make tool given: (1) we're primarily supporting Python projects, (2) the DSL is less crazy than Make, (3) it won't be as annoying for Windows users to install.

pjbull avatar Jul 25 '16 22:07 pjbull

+1 for allowing supporting DAG managers.

Make is great, but IMHO there are much better/modern pipeline frameworks. My favourite Airflow

victormartingarcia avatar May 11 '17 08:05 victormartingarcia

Here's a cookiecutter I found using snakemake: https://github.com/xguse/cookiecutter-snakemake-analysis-pipeline

jamesmyatt avatar Sep 06 '18 11:09 jamesmyatt