tinygo
tinygo copied to clipboard
tinygo-org
Using go-yaml/yaml fails with "(reflect.Type).Elem() called on map type"
It seems tinygo's reflect package is not fully compatible with golang's native reflect package.
I have a golang program. It can be compiled to binary with go build. I tried to use tinygo to compile it to WASM, I run into the following issue:
$ tinygo build -o set-ns.wasm -target wasi ./main.go
# sigs.k8s.io/kustomize/kyaml/internal/forked/github.com/go-yaml/yaml
/usr/local/lib/tinygo/src/reflect/type.go:385:15: (reflect.Type).Elem() called on map type
%0 = call i32 @"(reflect.rawType).elem"(i32 %t, i8* undef), !dbg !3004
traceback:
/usr/local/lib/tinygo/src/reflect/type.go:385:15:
%0 = call i32 @"(reflect.rawType).elem"(i32 %t, i8* undef), !dbg !3004
/usr/local/lib/tinygo/src/reflect/type.go:384:18:
%ret = call %runtime._interface @"(reflect.rawType).Elem"(i32 %unpack.int, i8* %1), !dbg !3000
/usr/local/google/home/mengqiy/golang/src/sigs.k8s.io/kustomize/kyaml/internal/forked/github.com/go-yaml/yaml:
%15 = call %runtime._interface @"interface:{Align:func:{}{basic:int},AssignableTo:func:{named:reflect.Type}{basic:bool},Bits:func:{}{basic:int},ChanDir:func:{}{named:reflect.ChanDir},Comparable:func:{}{basic:bool},ConvertibleTo:func:{named:reflect.Type}{basic:bool},Elem:func:{}{named:reflect.Type},Field:func:{basic:int}{named:reflect.StructField},FieldAlign:func:{}{basic:int},FieldByName:func:{basic:string}{named:reflect.StructField,basic:bool},Implements:func:{named:reflect.Type}{basic:bool},In:func:{basic:int}{named:reflect.Type},IsVariadic:func:{}{basic:bool},Key:func:{}{named:reflect.Type},Kind:func:{}{named:reflect.Kind},Len:func:{}{basic:int},MethodByName:func:{basic:string}{named:reflect.Method,basic:bool},Name:func:{}{basic:string},NumField:func:{}{basic:int},NumIn:func:{}{basic:int},NumMethod:func:{}{basic:int},NumOut:func:{}{basic:int},Out:func:{basic:int}{named:reflect.Type},PkgPath:func:{}{basic:string},Size:func:{}{basic:uintptr},String:func:{}{basic:string}}.Elem$invoke"(i8* %14, i32 %invoke.func.typecode, i8* undef), !dbg !3001
Stacktrace shows there is something wrong here.
The error message comes from here.
I just came here to report the same.
It is triggered by gopkg.in/yaml.v2 aka https://github.com/go-yaml/yaml
Here is output on my machine, tinygo 0.24:
ld.lld: warning: duplicate /export option: hypot
ld.lld: warning: duplicate /export option: nextafter
# gopkg.in/yaml.v2
C:\Users\marti\scoop\apps\tinygo\current\src\reflect\type.go:385:15: (reflect.Type).Elem() called on map type
<badref> = callException 0xc0000005 0x0 0x288637fbe10 0x7ff73b66c637
PC=0x7ff73b66c637
signal arrived during external code execution
runtime.cgocall(0x7ff73ac9d7d0, 0xc009f1f4e0)
C:/hostedtoolcache/windows/go/1.18.1/x64/src/runtime/cgocall.go:157 +0x4a fp=0xc009f1f4b8 sp=0xc009f1f480 pc=0x7ff73a9246ea
tinygo.org/x/go-llvm._Cfunc_LLVMDumpValue(0x28865592eb0)
_cgo_gotypes.go:5189 +0x52 fp=0xc009f1f4e0 sp=0xc009f1f4b8 pc=0x7ff73ab52cf2
tinygo.org/x/go-llvm.Value.Dump.func1({0x1?})
C:/Users/runneradmin/go/pkg/mod/tinygo.org/x/[email protected]/ir.go:712 +0x3f fp=0xc009f1f518 sp=0xc009f1f4e0 pc=0x7ff73ab65c5f
tinygo.org/x/go-llvm.Value.Dump({0xc011a00020?})
C:/Users/runneradmin/go/pkg/mod/tinygo.org/x/[email protected]/ir.go:712 +0x19 fp=0xc009f1f530 sp=0xc009f1f518 pc=0x7ff73ab65bf9
main.printCompilerError(0x7ff73f06d910, {0x7ff73f0da920?, 0xc010762e00?})
D:/a/tinygo/tinygo/main.go:1170 +0x4d8 fp=0xc009f1f7c8 sp=0xc009f1f530 pc=0x7ff73ac8e4f8
main.handleCompilerError({0x7ff73f0da920?, 0xc010762e00?})
D:/a/tinygo/tinygo/main.go:1203 +0x37 fp=0xc009f1f7f0 sp=0xc009f1f7c8 pc=0x7ff73ac8ee17
main.main()
D:/a/tinygo/tinygo/main.go:1511 +0x20ea fp=0xc009f1ff80 sp=0xc009f1f7f0 pc=0x7ff73ac916aa
runtime.main()
C:/hostedtoolcache/windows/go/1.18.1/x64/src/runtime/proc.go:250 +0x1fe fp=0xc009f1ffe0 sp=0xc009f1ff80 pc=0x7ff73a959d9e
runtime.goexit()
C:/hostedtoolcache/windows/go/1.18.1/x64/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc009f1ffe8 sp=0xc009f1ffe0 pc=0x7ff73a9854e1
rax 0x53
rbx 0x288637fbe00
rcx 0x920efff1f8
rdi 0x920efff1e0
rsi 0x0
rbp 0xc009f1f4a8
rsp 0x920efff1a0
r8 0x60
r9 0x920effec60
r10 0x0
r11 0x246
r12 0x28865592eb0
r13 0x920efff4e0
r14 0x920efff200
r15 0x920efff1f8
rip 0x7ff73b66c637
rflags 0x10246
cs 0x33
fs 0x53
gs 0x2b
tinygo 0.23:
ld.lld: warning: duplicate /export option: hypot
ld.lld: warning: duplicate /export option: nextafter
# gopkg.in/yaml.v2
C:\Users\marti\scoop\apps\tinygo\current\src\reflect\type.go:385:15: (reflect.Type).Elem() called on map type
%0 = call i64 @"(reflect.rawType).elem"(i64 %t, i8* undef), !dbg !3987
traceback:
C:\Users\marti\scoop\apps\tinygo\current\src\reflect\type.go:385:15:
%0 = call i64 @"(reflect.rawType).elem"(i64 %t, i8* undef), !dbg !3987
C:\Users\marti\scoop\apps\tinygo\current\src\reflect\type.go:384:18:
%ret = call %runtime._interface @"(reflect.rawType).Elem"(i64 %unpack.int, i8* %1), !dbg !3983
C:\Users\marti\go\pkg\mod\gopkg.in\[email protected]:
%6 = call %runtime._interface @"interface:{Align:func:{}{basic:int},AssignableTo:func:{named:reflect.Type}{basic:bool},Bits:func:{}{basic:int},ChanDir:func:{}{named:reflect.ChanDir},Comparable:func:{}{basic:bool},ConvertibleTo:func:{named:reflect.Type}{basic:bool},Elem:func:{}{named:reflect.Type},Field:func:{basic:int}{named:reflect.StructField},FieldAlign:func:{}{basic:int},FieldByName:func:{basic:string}{named:reflect.StructField,basic:bool},Implements:func:{named:reflect.Type}{basic:bool},In:func:{basic:int}{named:reflect.Type},IsVariadic:func:{}{basic:bool},Key:func:{}{named:reflect.Type},Kind:func:{}{named:reflect.Kind},Len:func:{}{basic:int},MethodByName:func:{basic:string}{named:reflect.Method,basic:bool},Name:func:{}{basic:string},NumField:func:{}{basic:int},NumIn:func:{}{basic:int},NumMethod:func:{}{basic:int},NumOut:func:{}{basic:int},Out:func:{basic:int}{named:reflect.Type},PkgPath:func:{}{basic:string},Size:func:{}{basic:uintptr},String:func:{}{basic:string}}.Elem$invoke"(i8* %invoke.func.value, i64 %invoke.func.typecode, i8* undef), !dbg !3984
It is triggered by gopkg.in/yaml.v2 aka https://github.com/go-yaml/yaml
The issue I have is triggered by this library too. We are using a forked version of https://github.com/go-yaml/yaml. But I believe there is same issue with gopkg.in/yaml.v2 and gopkg.in/yaml.v3.
Just tried with the new release v0.24.0 and got a similar error.
$ tinygo build -o set-ns.wasm -target wasi ./main.go
# sigs.k8s.io/kustomize/kyaml/internal/forked/github.com/go-yaml/yaml
/usr/local/lib/tinygo/src/reflect/type.go:385:15: (reflect.Type).Elem() called on map type
<badref> = callfatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4f86cb9]
runtime stack:
runtime.throw({0x53b25d1?, 0x6583748?})
/opt/hostedtoolcache/go/1.18.1/x64/src/runtime/panic.go:992 +0x71
runtime.sigpanic()
/opt/hostedtoolcache/go/1.18.1/x64/src/runtime/signal_unix.go:802 +0x3a9
goroutine 1 [syscall]:
runtime.cgocall(0xe5e0f0, 0xc00cc774b8)
/opt/hostedtoolcache/go/1.18.1/x64/src/runtime/cgocall.go:157 +0x5c fp=0xc00cc77490 sp=0xc00cc77458 pc=0xaff23c
tinygo.org/x/go-llvm._Cfunc_LLVMDumpValue(0x7fb40c0a9c40)
_cgo_gotypes.go:5196 +0x45 fp=0xc00cc774b8 sp=0xc00cc77490 pc=0xd15985
tinygo.org/x/go-llvm.Value.Dump.func1({0x1?})
/home/runner/go/pkg/mod/tinygo.org/x/[email protected]/ir.go:712 +0x3f fp=0xc00cc774f0 sp=0xc00cc774b8 pc=0xd26f1f
tinygo.org/x/go-llvm.Value.Dump({0xc00d4397a0?})
/home/runner/go/pkg/mod/tinygo.org/x/[email protected]/ir.go:712 +0x19 fp=0xc00cc77508 sp=0xc00cc774f0 pc=0xd26eb9
main.printCompilerError(0x53bb058, {0x541c980?, 0xc00ba8f730?})
/home/runner/work/tinygo/tinygo/main.go:1170 +0x4d8 fp=0xc00cc777c0 sp=0xc00cc77508 pc=0xe3bed8
main.handleCompilerError({0x541c980?, 0xc00ba8f730?})
/home/runner/work/tinygo/tinygo/main.go:1203 +0x37 fp=0xc00cc777e8 sp=0xc00cc777c0 pc=0xe3c7f7
main.main()
/home/runner/work/tinygo/tinygo/main.go:1444 +0x3b9a fp=0xc00cc77f80 sp=0xc00cc777e8 pc=0xe40b3a
runtime.main()
/opt/hostedtoolcache/go/1.18.1/x64/src/runtime/proc.go:250 +0x212 fp=0xc00cc77fe0 sp=0xc00cc77f80 pc=0xb316d2
runtime.goexit()
/opt/hostedtoolcache/go/1.18.1/x64/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00cc77fe8 sp=0xc00cc77fe0 pc=0xb5eca1
This is not unexpected. Perhaps the warning about reflect in https://tinygo.org/docs/reference/lang-support/ should be stronger. It is hard to predict when tinygo's reflect will support maps better. https://github.com/tinygo-org/tinygo/pull/2640 might be a step towards a fix, but it's a big job.
https://tinygo.org/docs/reference/lang-support/stdlib/ shows many standard library packages fail to pass tests due to reflect being incomplete in tinygo. https://github.com/tinygo-org/tinygo/issues/2566 is related.
And running https://github.com/go-yaml/yaml's tests in tinygo fails with:
../../go/pkg/mod/gopkg.in/[email protected]/check.go:56:30: receiver.Method undefined (type reflect.Value has no field or method Method)
../../go/pkg/mod/gopkg.in/[email protected]/check.go:56:57: receiver.Type().Method undefined (type reflect.Type has no field or method Method)
As for the specific panic in this case, the we do not encode the map's full type into a side-table in the binary that would allow us to get the key and element types.
The go-yaml/yaml tests won't pass because they rely on Method functions, which we don't support for reflect. However basic yaml support should be working on the last few PRs are merged. You can try it now by checking out this branch: https://github.com/dgryski/tinygo/tree/dgryski/reflect-all-fixes-3
Running the go-yaml/yaml example from https://github.com/go-yaml/yaml#example gives:
~/go/src/github.com/dgryski/bug/yml $ tinygo run main.go
--- t:
{Easy! {2 [3 4]}}
--- t dump:
a: Easy!
b:
c: 2
d: [3, 4]
--- m:
map[a:Easy! b:map[c:2 d:[3 4]]]
--- m dump:
a: Easy!
b:
c: 2
d:
- 3
- 4
Thank you for working on this. I want to use tinygo to compile my app, but it heavily depends on that yaml library. Thank you for all your effort. I've been watching this thread, waiting to see this.
