cascalog-graph
cascalog-graph copied to clipboard
jeroenvandijk
Add better failure information
Currently Cascalog Checkpoint reports errors as "ERROR checkpointed-workflow (Thread-23): Component failed" and you get the stacktrace. Many times this stacktrace is enough, but sometimes it is hard to see which step actually failed. In both causes it would be easier if the state of the graph would be emitted to a file so it could easier be debugged. This could for example be visualized through Graphviz.
This feature can possibly be combined with #4 and added timing information as well. Both need the implemented at the Cascalog Checkpoint level
We can use Graphviz through Canviz a js library to render dot files in the browser. For example, when a workflow fails a dot file could be created and uploaded to the same location as where the log files and the job jar are located. The link to the html page with this rendered graph could be in the error so it is only one copy-paste away.
When using Canviz, we can also try to format the graph like the images in this README https://github.com/pahen/node-madge/
Add :serial-order? true as option to run in serial mode instead of parallel mode
Add names to failing steps