WASM: stack overflow when unmarshalling JSON

[cryi]

Environment

TinyGo version: 0.29.0 (with #3890 patch) Environment: WASM

Description

We've encountered a stack overflow bug specifically when targeting WASM using TinyGo. Interestingly, we were unable to replicate the issue on desktop targets.

Minimal Reproducible Example

package main

import (
	"encoding/json"
	"fmt"
	"syscall/js"
)

var (
	terminateChannel = make(chan struct{})
)

type Person interface {
}

type JsonResponse struct {
	Generated []Person `json:"generated"`
}

func main() {
	mre := js.Global().Get("Object").New()
	js.Global().Set("mre", mre)
	mre.Set("trigger", js.FuncOf(trigger))
	<-terminateChannel
}

func trigger(this js.Value, args []js.Value) interface{} {
	// removing any level from the JSON below solves the stack overflow issue
	data := `{
		"generated": [
			{
				"metadata": {
					"person_records": {
						"errors": [
							{
								"with": {
									"int": "1004"
								}
							}
						]
					}
				}
			}
		]
	}`
	var response JsonResponse
	err := json.Unmarshal([]byte(data), &response)
	if err != nil {
		panic(err)
	}

	fmt.Println(response) // the program prints this and then experiences a stack overflow
	terminateChannel <- struct{}{}
	return nil
}

From JavaScript, it's invoked as:

const tezpay = globalThis.mre
tezpay.trigger()

Steps to reproduce:

1. compile the provided code using tinygo targeting WASM. (tinygo clean && tinygo build -o mre.wasm -target wasm -no-debug mre.go)
2. Include https://github.com/tinygo-org/tinygo/blob/release/targets/wasm_exec.js in your index.html
3. Run the resulting WASM binary.

async function wasmBrowserInstantiate(wasmModuleUrl, importObject) => {
	let response = undefined;

	if (!importObject) {
		importObject = {
			env: {
				abort: () => console.log("Abort!")
			}
		};
	}


	if (WebAssembly.instantiateStreaming) {
		response = await WebAssembly.instantiateStreaming(
			fetch(wasmModuleUrl),
			importObject
		);
	} else {
		const fetchAndInstantiateTask = async () => {
			const wasmArrayBuffer = await fetch(wasmModuleUrl).then(response =>
				response.arrayBuffer()
			);
			return WebAssembly.instantiate(wasmArrayBuffer, importObject);
		};
		response = await fetchAndInstantiateTask();
	}

	return response;
};

const go = new Go();
const importObject = go.importObject;
// adjust path to built WASM as needed
const wasmModule = await wasmBrowserInstantiate("/wasm/mre.wasm", importObject);
go.run(wasmModule.instance);

wasmModule.trigger() // triggers stack overflow

Expected Behavior

The Go code should successfully unmarshal the JSON data without any issues. (GO is able to unmarshal without stack overflow)

Current Behavior

Upon invoking the trigger, the program does print the unmarshaled result. However, it subsequently crashes due to a stack overflow.

Notes

Parsing in main wont trigger stack overflow so the issue may be related to the channels or JS interop.

Any help on this would be greatly appreciated!

[dgryski]

I tweaked the binary to run without javascript (but still wasm):

package main

import (
	"encoding/json"
	"fmt"
)

var (
	terminateChannel = make(chan struct{})
)

type Person interface {
}

type JsonResponse struct {
	Generated []Person `json:"generated"`
}

func main() {
	go trigger()
	<-terminateChannel
}

func trigger() interface{} {
	// removing any level from the JSON below solves the stack overflow issue
	data := `{
		"generated": [
			{
				"metadata": {
					"person_records": {
						"errors": [
							{
								"with": {
									"int": "1004"
								}
							}
						]
					}
				}
			}
		]
	}`
	var response JsonResponse
	err := json.Unmarshal([]byte(data), &response)
	if err != nil {
		panic(err)
	}

	fmt.Println(response) // the program prints this and then experiences a stack overflow
	terminateChannel <- struct{}{}
	return nil
}

Then as we can see, it runs fine with go and tinygo, but tinygo -target=wasi fails. I believe this is due to a stack overflow, and you can see that increasing the stack size solves the problem. This is becoming a common failure case for people wanting to parse json.

~/go/src/github.com/dgryski/bug/json2 $ go run main.go
{[map[metadata:map[person_records:map[errors:[map[with:map[int:1004]]]]]]]}
~/go/src/github.com/dgryski/bug/json2 $ tinygo run main.go
{[map[metadata:map[person_records:map[errors:[map[with:map[int:1004]]]]]]]}
~/go/src/github.com/dgryski/bug/json2 $ tinygo run -target=wasi main.go
{[map[metadata:map[person_records:map[errors:[map[with:map[int:1004]]]]]]]}
panic: runtime error: nil pointer dereference
Error: failed to run main module `/var/folders/5b/_hr1d1fd3qn4t9vtfx5p61wm0000gp/T/tinygo1150388317/main`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x23082 - runtime.abort
                           at /Users/dgryski/go/src/github.com/tinygo-org/tinygo/src/runtime/runtime_tinygowasm.go:70:6              - runtime.runtimePanicAt
                           at /Users/dgryski/go/src/github.com/tinygo-org/tinygo/src/runtime/panic.go:71:7
           1: 0x741c - runtime.nilPanic
                           at /Users/dgryski/go/src/github.com/tinygo-org/tinygo/src/runtime/panic.go:134:16
           2: 0x61ec7 - main.trigger
                           at /Users/dgryski/go/src/github.com/dgryski/bug/json2/main.go:51:2
           3: 0x24ce7 - <goroutine wrapper>
                           at /Users/dgryski/go/src/github.com/dgryski/bug/json2/main.go:20:2
           4:  0x7a8 - tinygo_launch
                           at /Users/dgryski/go/src/github.com/tinygo-org/tinygo/src/internal/task/task_asyncify_wasm.S:59
           5: 0x242c3 - (*internal/task.Task).Resume
                           at /Users/dgryski/go/src/github.com/tinygo-org/tinygo/src/internal/task/task_asyncify.go:109:17              - runtime.scheduler
                           at /Users/dgryski/go/src/github.com/tinygo-org/tinygo/src/runtime/scheduler.go:236:11              - runtime.run
                           at /Users/dgryski/go/src/github.com/tinygo-org/tinygo/src/runtime/scheduler_any.go:28:11              - _start
                           at /Users/dgryski/go/src/github.com/tinygo-org/tinygo/src/runtime/runtime_wasm_wasi.go:21:5
    2: wasm trap: wasm `unreachable` instruction executed
error: failed to run compiled binary /var/folders/5b/_hr1d1fd3qn4t9vtfx5p61wm0000gp/T/tinygo1150388317/main: exit status 134
~/go/src/github.com/dgryski/bug/json2 $ tinygo run -target=wasi -stack-size=32KB main.go
{[map[metadata:map[person_records:map[errors:[map[with:map[int:1004]]]]]]]}

[deadprogram]

I suspect this issue is solved for wasi and wasm in the latest dev branch by increasing the default stack size.

Can you retest with the latest dev?

[cryi]

Will do @deadprogram

[cryi]

@deadprogram, I've tested with the patch on the dev branch. I didn't use the latest dev as I had trouble compiling it, but I checked out to the commit of that PR - ff32fbbb4ff266f5f29080479a29d941cad2a57e. Unfortunately, 32K doesn't seem sufficient for our production needs. I was able to work with our WASM using 64K, but it's hard to determine with certainty whether that's adequate for production.

Wouldn't it be more reasonable to opt for a larger default? I've looked into other implementations, and it's common to find values ranging from 0.5M to 5M.

[deadprogram]

From the comment by @dkegel-fastly in the Slack channel, seems like 64k is the way.

https://github.com/emscripten-core/emscripten/pull/18191

I will make another PR doing bumping up to 64k.

[dkegel-fastly]

Another data point: our real world wasi app needs a stack size of 48KB. That's one more little bit of evidence 64KB is a good default stack size for wasi.

[dgryski]

WASI default stack size is now 64K. I think we can tag this as "next-release".

[deadprogram]

Completed with v0.31.0 so now closing. Thank you!