Feature request: DOT (or similar) graph generation from WDL

[pshapiro4broad]

I'm not sure if this falls under the umbrella of things miniwdl does, but it would be nice if it could generate a graph file (using DOT or some similar format) from an input WDL. This feature was supported by cromwell's womtool graph command but it has never worked for WDL 1.0. I'm sure miniwdl has all the data needed to generate such a graph, and someone could use it as a library to do this as well.

[mlin]

Thanks for this suggestion; I think it would make a nice example program for the documentation (#228), should be pretty simple yet actually useful in its own right.

[mlin]

building block -- https://github.com/xflr6/graphviz

[jdidion]

I will have some time to take this on in January, if someone else hasn't grabbed it by then. This will be hugely useful since WOMtool graph doesn't yet support WDL 1.0.

[mlin]

I've checked in a crude first cut at this https://github.com/chanzuckerberg/miniwdl/blob/master/examples/wdlviz.py

The results look pretty bad, because I'm unfamiliar with graphviz and generally lacking in the visual design talents. The good news is that it's only about 100 SLOC since it just has to project miniwdl's own model of the workflow dependency graph. So, I'm hoping somebody with more viz talent will be able to run with this and refine it a lot further :sweat_smile:

PRs welcome!!

Examples- install miniwdl & graphviz (using pip3 or conda) and

$ ./wdlviz.py https://raw.githubusercontent.com/mlin/DeepVariant-GLnexus-WDL/master/wdl/DeepVariant_GLnexus.wdl

$ ./wdlviz.py https://raw.githubusercontent.com/gatk-workflows/gatk4-germline-snps-indels/master/joint-discovery-gatk4-local.wdl

cc @vdauwera @geoffjentry

[pshapiro4broad]

For my usage, here's an example of the level of detail that would be helpful for me. This is the output of "womtool graph" on an older version of the GATK single sample pipeline. The graph shows dependencies, scatter/gather, leaf nodes and root nodes. I think it also shows conditionals. It doesn't show inputs and outputs, which removes a lot of complexity.

PairedSingleSampleWf.autogenerated.dot.txt

[mlin]

Our new codelab cleans it up a bit, worth a try. Hoping for someone to heroically swoop in and rescue the visual design =)

[kislyuk]

imho graphviz is just too dated for this.

Consider emitting mermaid-consumable markdown: https://mermaid-js.github.io/mermaid/

[mlin]

Consolidated from #322

We have a codelab on using the AST to create a crude graphviz visualization, https://miniwdl.readthedocs.io/en/latest/wdlviz.html and a slightly (not much) better version checked in the repo: https://github.com/chanzuckerberg/miniwdl/blob/master/examples/wdlviz.py We probably won't have bandwidth to fully flesh this out in the near future, but would definitely welcome if someone else is able to run with it!