docsonnet icon indicating copy to clipboard operation
docsonnet copied to clipboard

refactor: use jsonnet render engine in go binary

Open Duologic opened this issue 2 years ago • 1 comments

This PR removes the Go implementation of the render engine and replacing it by simply invoking the jsonnet code from doc-util in a Jsonnet VM.

This'll get rid of the diverging behavior in rendering engines and reduce toil on maintenance while still providing a binary for those that prefer that.

This also removes the option show the Package data as JSON, I believe they mostly existed to debug the Extract and Transform functions of the go code.

Considering the low number of lines (~100), I've also dropped everything in main.go.

Fixes #35 Replaces #37 Replaces #39


This PR takes a very similar approach as #39 but far more destructive as it doesn't intend to maintain the Go version.

In #39, @yorinasub17 suggests that it hits a max-stack-frames on (tf-libsonnet/hashicorp-aws), I was not able to reproduce that with this PR. Performance wise it was terribly slow, I've ran the same code with jrsonnet (rust version) and that was magnitudes faster (~40x) and didn't burn up my CPU. So perhaps it would be beneficial to use the jrsonnet C bindings here instead of the native Go implementation (next PR?).

➜ time go run . -o b -J vendor ./vendor/github.com/tf-libsonnet/hashicorp-aws/5.x/main.libsonnet
go run . -o b -J vendor   500.89s user 8.70s system 167% cpu 5:03.56 total

➜ time jrsonnet -S -J vendor -c -m d --exec "(import 'doc-util/main.libsonnet').render(import 'github.com/tf-libsonnet/hashicorp-aws/5.x/main.libsonnet')"
jrsonnet -S -J vendor -c -m d --exec   13.19s user 0.84s system 99% cpu 14.158 total

➜ time jsonnet -S -J vendor -c -m d --exec "(import 'doc-util/main.libsonnet').render(import 'github.com/tf-libsonnet/hashicorp-aws/5.x/main.libsonnet')"
jsonnet -S -J vendor -c -m d --exec   473.37s user 10.30s system 164% cpu 4:53.24 total

I've also resolved the ./render.libsonnet import with a simple strings.ReplaceAll on the main.libsonnet, embedding the whole thing.

Duologic avatar Sep 20 '23 21:09 Duologic

While running this against k8s-libsonnet, it surfaces a few changes that should be considered, I'll circle back to this when I have bit more time.

Duologic avatar Sep 21 '23 11:09 Duologic