Plugin system and "renderers"

[dlespiau]

Wouldn't it be nice if we could bring into jk objects from external sources? For instance we could render a helm chart, bring the result into jk as an array of objects, modify objects as we see fit and write the final result to files.

This PR introduces plugins and the first type of plugins: Renderer. A Renderer takes a js object as input and returns a js value. This PR includes a test Echo Renderer plugin that simply outputs the input it receives.

Plugins are identified by a json document with some metadata about the plugin, including where to find plugin binaries for the current OS/architecture.

echo.json

{
  "name": "echo",
  "version": "0.1.0",
  "binaries": {
    "linux-amd64": "jk-plugin-echo"
  }
}

simple usage example

import { render } from '@jkcfg/std/render';

render('echo.json', { message: "success" }).then(r => std.log(r.message));

TODO:

• [x] Retrofit a design doc
• [x] ~Support downloading the plugin description and the plugin binaries from http URLs~ decided not to do this.
• [x] Validate the system by implementing a helm plugin in @jkcfg/kubernetes (https://github.com/jkcfg/kubernetes/pull/55)
• [ ] (maybe) Port the jk <-> go communication to the new internal RPC method?
• [ ] (maybe) Port the plugins to gRPC instead of go RPC so we support plugins written in any language and not just go?
• [ ] Explore a bit more a --debug-plugin option that would make plugin output to stderr
• [x] Turn the echo plugin into a test plugin
• [ ] Add a test returning an error from the test plugin
• [ ] Add options to the RPC (to be future proof, eg. pass the directory hash where the binary is)
• [ ] Remove the need for the plugin.json file
• [ ] Add a way to define a function as a Renderer

Updates: #133

[squaremo]

• [ ] Retrofit a design doc

Looking forward to this :-)

• [ ] Support downloading the plugin description and the plugin binaries from http URLs

I would advise leaving this for later, in order to

1. not bite off too much in one go
2. defer decisions that will have repercussions beyond plugins

To expand on 2., I will admit here to be very nervous about anything that looks like packaging and distribution, and anything that does automatic downloads. The first bout of nerves is that it will effectively determine how packaging works for everything to do with jk, but without including all the other stuff in the consideration. The second is because automatically downloading stuff introduces all sorts of concerns about trust and repeatability.