_js: ideas to improve the API

[mvdan]

There have been a few discussions in other issues, such as @rob-myers' suggestions in #298.

As a start:

• Replace syntax.NodeType with a field in each node, such as n.type or n.type().
• Provide a way to fully clean the JS objects to remove the internal GopherJS state (similar to shfmt -tojson)

[rob-myers]

Would you mind updating the npm repo? I'm eager to try Parser.{Interative,Incomplete}.

[mvdan]

@rob-myers I'm not sure I follow - I published 0.2.0 a few days ago precisely to include those new APIs, among other things. See f6fe14d882ec30a63b53c40fe27fc98142ff8d40.

Also, you're right that the build instructions are somewhat vague. I'll add some more docs.

[mvdan]

Provide a way to fully clean the JS objects to remove the internal GopherJS state

By the way, if we do this, the JS objects would no longer be usable to the mvdan-sh package. Without the internal state, the JS objects can't be translated back into Go-like objects that the transpiled Go source can work with.

The functionality can still be useful, but we just need to keep that in mind and document it well.

[rob-myers]

I'm biased because I want to work purely with the parse-tree. It could be useful to those who want to do their own analysis/transformations/interpreter.

Wouldn't it be possible to reconstruct the source from the JSON parse-tree?

[mvdan]

It should be possible, but then one's code would look like:

var f1 = parser.Parse(...)
var f2 = syntax.ToJSON(f1)
// use and modify f2
f1 = syntax.FromJSON(f2)
printer.Print(..., f1)

Option B would be for all JS APIs to return this JSON, with the caveat that methods/funcs would be lost. We could re-insert them as func fields, which I think would work.

[rob-myers]

I'm trying to understand how you handle brace-expansion.

Parsing {echo,foo,bar} yields a single Word with a single literal. But parsing {echo,foo,$( echo bar )} yield a single Word with three parts: a literal {echo,foo,, a command substitution, and a literal }.

It makes sense that Parser does not perform brace-expansion i.e. does not transform the code. The interpreter can first interpret Word.Parts, concatenate, and then perform brace-expansion?

[mvdan]

Brace expansion isn't static, so yes - the parser can't perform it. There's an API for that: https://godoc.org/mvdan.cc/sh/expand#Braces

I have yet to expose this in the JS API, however. I can do that today or next week.

[rob-myers]

No need. I just wanted a sanity check, thanks.

[mvdan]

Looks like gopherjs is alive again, and it has support for Go 1.16.x. I'll try to use that to release a new version of the npm package soon, as the existing v0.5.0 is lagging a bit behind.

I've also been playing with Go's js/wasm port lately, and it seems like we could also use it to expose the parser. Not exactly in the same way, but pretty similar. Does anyone use the npm package in an environment that has javascript, but not wasm? The two main environments I can think of are Node and a modern browser, and both support wasm.

[rob-myers]

A new release of the npm module would be great, if possible. Are there any particular breaking changes to be aware of?

I will update my typings:

https://github.com/rob-myers/rob-myers.github.io/blob/three-cli/types/mvdan-sh/index.d.ts

[mvdan]

I don't currently plan any breaking changes, right now. Though I imagine some changes will be breaking by nature at some point, like https://github.com/mvdan/sh/issues/298.

[mvdan]

@rob-myers what environment do you run the mvdan-sh JS code in? I'm thinking of replacing the big blob of JS with a thin JS wrapper around webassembly, which would improve performance and allow dropping the gopherjs dependency - Go supports js/wasm now. But I wonder if any of my downstream users are on an environment where loading and running webassembly is a problem.

[rob-myers]

I run it in the browser, so web assembly is fine by me. Thanks.

[joscha]

I've also been playing with Go's js/wasm port lately, and it seems like we could also use it to expose the parser.

Do you think it would be possible to expose the formatter as well? I am thinking in terms of writing a shellfmt plugin for https://github.com/dprint/dprint that is not a process plugin (e.g. calling out to the shellfmt bianry) but instead making it a native wasm plugin for dprint (which is the preferred way to write plugins for dprint). The idea is to produce a wasm-based shim for dprint and then use a wasm shellfmt binary from it.

[mvdan]

@joscha apologies for the late reply. The formatter is not much different from the parser in terms of Go code, so it should be possible to compile it to JS or Wasm in a similar manner.

For some more general context, see https://github.com/mvdan/sh/issues/760 :)