neuron icon indicating copy to clipboard operation
neuron copied to clipboard

Support for custom Pandoc filters

Open srid opened this issue 5 years ago • 6 comments

Suppose we use Pandoc's JSON filters, then add a new metadata property to YAML frontmatter such as filter: foo.py, and have neuron look for the script file named ./neuron/filters/foo.py. Then, that zettel's Pandoc representation can be passed as JSON to the script's stdin, which in turn spits out the modified Pandoc AST JSON in stdout, for neuron to use.

The goal is to allow the user to customize the what gets rendered finally. Such as, say, evaluating code blocks and injecting the results back into the Pandoc AST.

We could also support builtin filters (in Haskell), and make neuron's link processing just another Pandoc filter (which is what it is anyway, in essense).

srid avatar Aug 27 '20 20:08 srid

A note on performance: if you have hundreds of notes using a filter, this would slow down site generation due to having to spawn the process that many times. So we could consider having the script consume input JSON infinite number of times in one spawn session. So neuron will spawn that script ahead, and then feed it the JSON of every zettel it needs transformed, while ending it only at the end (or not at all).

srid avatar Aug 27 '20 20:08 srid

Might have to do #321 first, to avoid unnecessary reruns.

srid avatar Aug 27 '20 20:08 srid

We can go one step further and add a plugin mechanism to neuron, where a plugin can determine a dynamic set of pages (or whatever) to generate based some structured data in notes.

Plugins can include:

  • [ ] Task management: Looks for list items in format of todotxt and aggregates them in one task-dashboard.html
  • [ ] Spaced repetition: No HTML generation, but perhaps a CLI tool that collects flash cards from neuron notes, with some local cache db for tracking learning.
  • [ ] Smart review: identify notes to review based on last_updated date, etc. (related somewhat to spaced repetition)
  • [ ] Self-tracking: https://lobste.rs/s/ri5utx/syntax_for_self_tracking

srid avatar Sep 21 '20 01:09 srid

In addition to todotxt, the taskpaper format could be interesting. Maybe not the whole format, but the tags (obviously a confusing name within the context of neuron). I think that the taskpaper-mode package may have the best example about how the format could be used.

Not exactly related to this, but if you haven't looked at it, there may be some interesting synergies with jupytext. It also uses pandoc behind the scenes. It could be nice to add computations and plots to notes.

jcmkk3 avatar Sep 21 '20 04:09 jcmkk3

AIUI this would also be required to add custom TEX packages? I was eyeing chemfig, but I'd expect the problem to apply to any latex package. I am no latex expert..

smhendrickson avatar Oct 13 '20 14:10 smhendrickson

I would love to see this, and I can replace https://github.com/iamcco/markdown-preview.nvim

I'm currently adding this to head.html.

<script src="https://cdnjs.cloudflare.com/ajax/libs/viz.js/2.1.2/viz.js" integrity="sha512-vnRdmX8ZxbU+IhA2gLhZqXkX1neJISG10xy0iP0WauuClu3AIMknxyDjYHEpEhi8fTZPyOCWgqUCnEafDB/jVQ==" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/viz.js/2.1.2/full.render.js" integrity="sha512-1zKK2bG3QY2JaUPpfHZDUMe3dwBwFdCDwXQ01GrKSd+/l0hqPbF+aak66zYPUZtn+o2JYi1mjXAqy5mW04v3iA==" crossorigin="anonymous"></script>
<script>
window.addEventListener("load", function(){
  let viz = new Viz();
  for (let element of document.getElementsByClassName("graphviz")) {
    let parent = element.parentNode
    let pparent = parent.parentNode
    viz.renderSVGElement(element.textContent)
    .then(function(element) {
      pparent.replaceChild(element, parent)
    });
  }
});
</script>

aca avatar Nov 25 '20 06:11 aca