Improved aspect ratio layout for Diagrams

[CBroz1]

Feature Request

Problem

Diagrams of extensive schema become difficult to understand when nodes are displayed on only one line. Currently, horizontal level conveys which tables take primary keys from common upstream tables, but are illegible when embedded within README files. An optional input could dictate Diagram rigidity or use of vertical space.

Justification

Adding more legible diagrams would facilitate communication about novel schema.

Examples

Current diagram, taken from workflow-animal: Diagram created with unflatted -l 4:

Alternative Considerations

This StackOverflow post suggests using GraphViz's unflatten on dot files. For example:

mydiagram=dj.Diagram(subject).make_dot()
mydiagram.write(path='mydiagram_dotfile',format='dot')

unflatten -l 4 mydiagram_dotfile | dot -Tpng -o condensed_diagram.png

• DataJoint currently uses PyDot to achieve the dot format, but pydot does not appear to integrate unflatten.
• PyGraphViz appears to be an alternative and fully supports Unflatten.

Requirements

Implementation could include any of the following:

1. Adding more documentation about the above use of GraphViz outside of Python
2. Add PyGraphViz unflatten function as optional step between within make_png, dictated by an edge length argument ... if the two dependencies work together, which I have not found a solution for in quick testing.

from pygraphviz import unflatten
def make_png(self):
    if edge_len is not None:
         pygraphviz_obj = self.unflatten(f'-l {edge_len}')
         return io.BytesIO(pygraphviz_obj.create_png())
    else: return io.BytesIO(self.make_dot().create_png())

3. Remove dependency on PyDot in favor of PyGraphViz - likely unnecessary.
4. Dynamically animating dot file represenation for full no-code visualizations.

[Akshat977]

Hi @CBroz1 , This issue seems to be a good initiation of my journey towards contribution to FOSS projects. Can you please assign this to me?

[guzman-raphael]

Thanks for the detailed request @CBroz1! :clap: This makes a lot of sense; navigating a complex pipeline has been reported by many to be a difficult barrier when coming onboard to a new project. One idea that would help with that but also mitigate such aspect ratios is table tagging.

Perhaps a new feature of DJ tables could be that you can specify N number of tags as strings. This would allow you to group or collect certain steps or tables that are similar. Ultimately, this would allow us to integrate a mode in visualizing the diagrams that uses these layers so you could zoom in and zoom out to provide the context/complexity when needed. This unflatten option is interesting so it might provide a stopgap until then.

[guzman-raphael]

Thanks for your interest @Akshat977! We always appreciate any help you can provide. :smiley:

This enhancement request here could actually be addressed in a variety of ways but might be a bit more complex for the first time contributor. We welcome all PR's but I might suggest having a look at these good-first-issues. We've recently updated them and they should reflect some easier ones to get started.

Please let me know if any of those catch your eye and I can assign them to you.