factotum
factotum copied to clipboard
snowplow
A system to programmatically run data pipelines
In systems with variable system paths (like if we are running out of Docker) it can be useful to be able to template the command parameter so that local testing...
[crypto](https://crates.io/crates/crypto) has [no maintainers](https://rustsec.org/advisories/RUSTSEC-2016-0005.html), [sha2](https://crates.io/crates/sha2) is a good modern alternative to switch to.
We need to upgrade hyper to a modern version (part of the webhook plumbing). This may well result in the adoption of reqwest (built on hyper to make things easier).
If Factotum is running in a containerized environment it can be issued with SIGTERM - when this happens the application is killed instantly with no chance of sending out the...
When multiple steps in the DAG execute concurrently it can overload very limited dockerized solutions as it becomes difficult to predict how much compute will be needed at different stages...
We have started to use autogenerated datamodeling jobs. This relies on running "Generate and Run SQL Runner DAG" orchestrating task. The logs produced by it dependent on [Factotum output](https://github.com/snowplow/factotum/blob/702014704edd0d882515389eb6cd1b5efa85832f/src/main.rs#L165). Here's...
All missing variables that are defined in the Factfile are treated as empty strings - I would expect this to fail instead.