Add architecture diagram functionality

[judepayne]

Description

Hi @davidwatkins73 , I would be interested in adding enhanced Architecture diagram generation functionality to Waltz using my library dictim. Under the covers, dictim uses d2 to generate the final diagram. Would you be interested in this PR? I would probably need a bit of guidance :)

Resourcing

We intend to contribute this feature

[judepayne]

Simple examples of the type of Architecture pictures that could be automatically generated and displayed in Waltz:

[davidwatkins73]

Hi @judepayne - these look great. Couple of questions:

• Would the dictim be dynamically generated by Waltz and rendered dynamically ?
• ...or would users interactively build out the dictim to be saved and rendered semi-statically
• I was looking through the d2 spec and can't see if it would be possible to attach metadata
  • Thinking it may be useful to attach an asset code to an application ('xyz-123') which we can use to dynamically overlay data onto a diagram, and also to x-ref the contents of diagrams via a 'Bill-of-Materials'
    • This is similar to the approach we started with the built-in Waltz flow diagrams
• Some of the grouping constructs in the examples above don't really have analogous entities in Waltz. It would be a useful exercise for us to work out how much is 'model' data and how much is 'diagram' data.

Hoping to get some time later this week to give this a proper try-out and hopefully come up with some of my own answers to the above :)

[judepayne]

Hi @davidwatkins73 - lots of good questions:

• The user wouldn't know about dictim at all. It's a data format designed to be generated by some program - in contrast to d2 which is a text language designed to be written by hand. The dictim (just json) would be generated dynamically, sent to dictim.server which returns svg.
• re. metadata: d2/dictim are focused on expressing the diagram and so don't support metadata, but dictim.server exposes a higher level api 'graph' which does support metadata/supplemental data. The docs are here. I suggest we use the 'graph' api; it makes it very easy to express a boxes and arrows diagram. A quick example of dictim vs the graph api to demonstrate the metadata difference.. here's a shape (e.g. an application in Waltz) expressed in dictim:

["xyz-123" "eSpear"]

In a shape, the first item is its key, the second its label. And here's how the same thing could look using the 'graph' api:

{"nodes": [{"app-id": "xyz-123",
            "app-name": "eSpear",
             "owner": "George",
              "department": "Risk"}]
 "node->key" "app-id"}

Nodes is just a vector of maps, each describing an application (in the case of Waltz) - just any arbitrary map of key - values, e.g. 'facts' about the application which may/ may not be used in the diagram. 'node->key' is used to tell dictim.server how to uniquely identify that application (node). Similarly, each edge (i.e. line between/ data flow between two apps) is also expressed as just an arbitrary mode. Each must have a "src" and "dest" key but beyond that can have whatever you want. The 'graph' api tutorial I hope makes it clear.

• re. The 'model' data to 'diagram' data split I think becomes clear through the 'graph' api: nodes, edges, node->key keys are for your model data. The other keys node-specs, edge-specs, node->container, container->parent, container->attrs are about controlling how the diagram is rendered. I was thinking that the latter we can think of as forming a 'template' which is merged into the map holding the data keys.
• I was thinking that an easy way to start this might be to develop a diagram that replaces/ sits in a new tab alongside the boingy graph. Already there we have nodes & edges (or applications & data flows) data that is provided dynamically by Waltz as you navigate through its hierarchies. I had a quick Look at the code, enough to see that the data is provided in boingy graph by 'common' but haven't yet got the app up and running to see exactly what data we have for each node and each edge. The more facts about each, the better!
• re. grouping. In my example of the 'graph' api above, the first level of grouping is provided by an a "node->container" key to the map, e.g. we can choose "owner" or "department"; anything that's in the map for each node. Grouping up higher than that is the job of the "container->parent" key (which contains a hierarchy). e.g. George reports to Mike reports to Susan.

Look forward to your feedback. Ready to do some stuff on your suggestion. A new angular component to sit alongside boingy graph for example, if you agree that might be a good place to start.

[judepayne]

Forgot to mention that behind the scenes, the 'diagram spec' provided to the graph api - i.e. the map of all the keys I mentioned above - is just converted to dictim (and then to d2 and then to svg).

dictim is a rawer format though and nested so harder to form that the 'graph' api which breaks it all apart and flattens it out (e.g. separates the data from the rendering instructions, and hierarchies).

[judepayne]

Hi @davidwatkins73 and @JWoodland-Scott I have updated the graph-spec api to generally rationalise, add power and take account of your suggestions on our call. The new graph-spec api is in dictim (e.g. added to the command line tool - useful for playing) and also dictim.server, the containerised micro service version of dictim. Best place for docs is here on the dictim.server site. Please have a look and let me know your feedback.