neuron icon indicating copy to clipboard operation
neuron copied to clipboard

Embed Graphviz graphs

Open srid opened this issue 4 years ago • 15 comments

DOT language code blocks should render using d3-graphviz.

Similar to: https://forum.graphviz.org/t/render-graphviz-graphs-directly-in-your-posts/125

srid avatar Jun 11 '20 19:06 srid

Perhaps it would be a good idea to look into what d3.js has to offer, with something like pixi.js.

NullSense avatar Jun 17 '20 18:06 NullSense

Roam supports mermaid diagrams: https://roamresearch.com/#/app/help/page/kw78QlSZ6

srid avatar Jul 19 '20 17:07 srid

I've been working on a tool to 'transform' code blocks by piping their content through CLI utilities like graphviz, graph-easy, svgbob, ditaa, or any other program that accepts stdin and outputs to stdout (or even a JavaScript function).

2020-08-06_19-11-58_102:nx:2

2020-08-06_19-51-29_DP-3

I think it's an interesting approach as it enables all sorts of cool things beyond diagram generation. For example, you can execute Python code blocks and put the output into another code block, basically re-creating the basic functionality of a Jupyter notebook:

2020-08-06_19-40-16_102:nx:2

Maybe a general-purpose approach like this would be a good way to implement this functionality? There are definitely security concerns when it comes to executing arbitrary code, but I can imagine a number of ways to mitigate them (e.g. a configurable list of allowed executables).

b0o avatar Aug 07 '20 02:08 b0o

@b0o That's interesting.

There are two classes of code evaluation here:

  • A, that can be evaluated in the browser. Eg: using mermaid.js to generate flow charts in the browser from the Markdown code blocks
  • B, which requires shelling out to some interpreter and injecting the output in generated HTML

A is lightweight and safer (and can be integrated with Cerveau), whereas B is flexible - allowing one to use any tool installed.

A can be implemented in neuron itself (as a builtin plugin); and B can perhaps be done as some sort of general Pandoc filter? For example, you specify the executables to use as filter in neuron.dhall, and then neuron passes the AST to those programs, expecting them to output the final AST that gets used in generated html files.

srid avatar Aug 07 '20 22:08 srid

I think an integrated solution like mermaid.js would suffice most use cases and be much more portable, lightweight, and secure. I think that's the way to go for the most part. I do think an additional 'class B' solution would be quite valuable for the flexibility it offers, though.

I didn't know about Pandoc filters – I think that would be a great way to accomplish a 'class B' solution. Having a way to 'tap into' the rendering pipeline between Neuron and the output HTML would open up a lot of possibilities.

On a related note: since the tool I'm working on operates on Neuron's rendered HTML files and not the source markdown files, I have run into some issues dealing with Neuron/Rib/Shake thinking that the HTML files need to be re-generated after my script runs.

My guess (without investigating further) is that Neuron/Rib/Shake is caching the hashes or modification times of the HTML files (possibly in .shake/.shake.database?), which my program then invalidates by modifying those HTML files. The next time Neuron runs, the files my transfomer program modified are overwritten. Then, my program to needs to re-transform all of the files again, even if the change that triggered the Neuron run happened in an unrelated file.

I can think of a few ways to work around this, but if I could just hook into the rendering pipeline, e.g. with a Pandoc filter, I would assume that Neuron/Rib/Shake would cease to be confused about the modification made by my script and everyone would live happily ever after 😉.

b0o avatar Aug 08 '20 02:08 b0o

I came across https://excalidraw.com - which produces SVG of hand drawn diagrams. It is pretty cool. The web app has a "Copy to clipboard as SVG" feature, which you can past directly in a neuron markdown file and it will work. However, another option is to just paste the URL and let neuron do the replacement for us, kind of like: https://twitter.com/Vjeux/status/1257906664239333376

To make it self-contained, the diagram has to be put in the zettelkasten, perhaps as a excalidraw file (which apparently is JSON based).

srid avatar Aug 21 '20 17:08 srid

https://observablehq.com/@d3/force-directed-graph <- Very beautiful, interactive graphs.

NullSense avatar Aug 27 '20 08:08 NullSense

@b0o On my personal Zettelkasten, I hit upon a need for Pandoc filters myself (mainly to transform tasks lists based on inline tags to identify next actions), so it had me once again reflect on how this feature would look if implemented. So I've opened a feature request #363

srid avatar Aug 27 '20 20:08 srid

This feature can then be implemented using one of the existing Pandoc filters, i.e. https://github.com/hertogp/imagine

srid avatar Aug 27 '20 20:08 srid

Here's how I integrated chartjs, mermaid, graphviz. https://aca.github.io/c34ea231.html

aca avatar Nov 28 '20 15:11 aca

https://neuron.zettel.page/custom-head.html

srid avatar Nov 28 '20 23:11 srid

And here's how I embedded TikZJax.

zettelkasten/head.html

<!-- tikzjax -->
<link rel="stylesheet" type="text/css" href="https://tikzjax.com/v1/fonts.css">
<script src="https://tikzjax.com/v1/tikzjax.js"></script>
<script>
window.addEventListener("load", function(){
  for (let element of document.getElementsByClassName("tikzjax")) {
    let parent = element.parentNode;
    let pparent = parent.parentNode;
    let script = document.createElement('script'); script.type = 'text/tikz';
    let box = document.createElement('div');
    script.appendChild(document.createTextNode(element.textContent));
    box.appendChild(script)
    box.setAttribute("style","display:block;width:75%;text-align:'center';margin: 5px auto;");
    pparent.replaceChild(box, parent);
  }
});
</script>

zettelkasten/a-zettel.md

```tikzjax
\begin{tikzpicture}
   \draw (0,0) circle (1in);
\end{tikzpicture}
```
Screenshot 2020-12-08 at 14 48 43

0ihsan avatar Dec 08 '20 11:12 0ihsan

@ihsanturk Cool. Perhaps we can add it to https://neuron.zettel.page/custom-head.html

srid avatar Dec 09 '20 03:12 srid

Relevant: https://github.com/srid/neuron/issues/531#issuecomment-763027613 (a plugin that automatically injects the JS and inlines the code when it sees ![[Example1.graphviz]])?

srid avatar Jan 19 '21 18:01 srid

Would be very useful; part of a generalized approach to support external, structured data. Graphviz is the workhorse for visualizing RDF structures. Mermaid and similar visualizers are comparable (subsets). The key here - which you are doing - is generalizing service initiation. The equivalent model: the configuration.py setup in Sphinx where different Python libraries are initialized. A developer doesn't need to know the internals of the libraries; only the syntax to achieve a purpose.

jaygray0919 avatar Jan 19 '21 19:01 jaygray0919