stampede
stampede copied to clipboard
Teradata
Stampede is the workflow tool for *nix that Cthulhu intended. It uses make for dependencies, bash for scripting, and cron for scheduling.
The send_email tool assumes that email configured on the local machine. A more flexible email client would be nice that can connect to a preconfigured email server, etc.
Stampede reads make's output and logs it, providing a uniform source of the output data. However, it is noticeably slow when there is a rapid stream of output.
It would be nice add an example using Storm, where a continuous stream of processing runs in parallel to periodic tasks that consome output from Storm.
HCatalog has a Java API for listening for updates. This could be used to trigger steps in a workflow.