luigi icon indicating copy to clipboard operation
luigi copied to clipboard

[Contrib] Task trigger solution

Open redhog opened this issue 3 years ago • 0 comments

We use Luigi to run one-off data processing pipelines, and need a way to deploy and trigger these. As luigi doesn't include this, I've coded up a solution that lets you deploy code in virtualenv and trigger luigi tasks inside those virtualenvs using yaml files.

https://github.com/emerald-geomodelling/poltergust

Would there be an interest in including something like this in luigi.contrib? Are there other similar attempts I should be aware of?

redhog avatar Sep 23 '22 10:09 redhog