`go-ethereum` does not compile in tinygo

[xphoniex]

~/# go version
go version go1.18.3 linux/amd64
~/# tinygo version
tinygo version 0.23.0 linux/amd64 (using go version go1.18.3 and LLVM version 14.0.0)

Trying tinygo but can't manage to compile my small script:

~/test# tinygo build -o tiny_main main.go
# github.com/ethereum/go-ethereum/common
../go/pkg/mod/github.com/ethereum/[email protected]/common/debug.go:35:8: PrintStack not declared by package debug

Here's the debug.go:35:

package common

import (
        "fmt"
        "os"
        "runtime"
//      "runtime/debug"
        "strings"
)

// Report gives off a warning requesting the user to submit an issue to the github tracker.
func Report(extra ...interface{}) {
        fmt.Fprintln(os.Stderr, "You've encountered a sought after, hard to reproduce bug. Please report this to the developers <3 https://github.com/ethereum/go-ethereum/issues")
        fmt.Fprintln(os.Stderr, extra...)

        _, file, line, _ := runtime.Caller(1)
        fmt.Fprintf(os.Stderr, "%v:%v\n", file, line)

        //debug.PrintStack() // HERE

        fmt.Fprintln(os.Stderr, "#### BUG! PLEASE REPORT ####")
}

after commenting out the culprit I still can't compile:

~/test# tinygo build main.go
# github.com/ethereum/go-ethereum/rlp
../go/pkg/mod/github.com/ethereum/[email protected]/rlp/encode.go:349:44: val.Field(fields[lastField].index).IsZero undefined (type reflect.Value has no field or method IsZero)
../go/pkg/mod/github.com/ethereum/[email protected]/rlp/unsafe.go:31:15: v.UnsafeAddr undefined (type reflect.Value has no field or method UnsafeAddr)
../go/pkg/mod/github.com/ethereum/[email protected]/rlp/unsafe.go:32:12: cannot use length (variable of type int) as uintptr value in assignment
../go/pkg/mod/github.com/ethereum/[email protected]/rlp/unsafe.go:33:12: cannot use length (variable of type int) as uintptr value in assignment

Is this normal?

[dkegel-fastly]

That Fprintln appears to be using reflection, which is not well supported under tinygo yet.

go-etherium is a pretty big package with lots of dependencies, and it's not surprising it has some problems under tinygo.

[xphoniex]

That Fprintln appears to be using reflection, which is not well supported under tinygo yet.

Hmm... Fprintln is not the issue here, debug.PrintStack() is causing the error, which I then commented out and re-tried but got even more errors.

[dkegel-fastly]

Right, well, the other errors are the hard ones, and they're from go-ethereum hitting holes in tinygo.

The first one, type reflect.Value has no field or method IsZero, is part of the general "reflect is not fully implemented" problem. It's going to be a while before that is fixed, and there is no timeline for that fix.

Getting go-ethererum working with tinygo as it is now may be possible, but would require making a number of changes to go-ethereum, which is probably more work than you want to do yourself. See https://github.com/vugu/vugu/pull/219 for an example of another largish project that did it.

[dkegel-fastly]

IsZero ( https://github.com/tinygo-org/tinygo/pull/3300 ) landed in the dev branch of tinygo, so you may want to try again and see what the remaining errors are.

[dgryski]

The issue in unsafe.go is code that is inherently non-portable. You will need tinygo-specific version of that file / function.

Edit: looks like there is already a safe.go version; you just need to fix the build tags to include tinygo.

[xphoniex]

just built tinygo from dev branch and still can't compile:

$ go version
go version go1.18.3 linux/amd64
$ tinygo version
tinygo version 0.27.0-dev linux/amd64 (using go version go1.18.3 and LLVM version 14.0.6)

on build:

$ tinygo build -o tiny_main main.go
# github.com/ethereum/go-ethereum/crypto/secp256k1
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult.h:15:28: unknown C type: secp256k1_ge_storage[] (libclang type kind IncompleteArray)
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult.h:17:28: unknown C type: secp256k1_ge_storage[] (libclang type kind IncompleteArray)
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/include/secp256k1.h:43:16: struct_secp256k1_context_struct defined previously at /home/user/.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/./libsecp256k1/src/secp256k1.c:49:8 with a different type

also tried for arduino:

$ tinygo build -o zzz -target arduino
# github.com/ethereum/go-ethereum/crypto/secp256k1
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen.h:26:34: error: array is too large (64 elements)
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/secp256.go:29:10: note: in file included from secp256.go!cgo.c:20:
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/secp256k1.c:16:10: note: in file included from /home/user/.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/./libsecp256k1/src/secp256k1.c:16:
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen_impl.h:12:10: note: in file included from /home/user/.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen_impl.h:12:
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:153:19: error: shift count >= width of type
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/secp256.go:29:10: note: in file included from secp256.go!cgo.c:20:
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/secp256k1.c:16:10: note: in file included from /home/user/.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/./libsecp256k1/src/secp256k1.c:16:
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen_impl.h:13:10: note: in file included from /home/user/.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen_impl.h:13:
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:33:21: note: expanded from macro 'BE32'
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:153:19: error: shift count >= width of type
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:33:32: note: expanded from macro 'BE32'
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:153:19: error: shift count >= width of type
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:33:44: note: expanded from macro 'BE32'
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:153:19: error: shift count >= width of type
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:33:68: note: expanded from macro 'BE32'
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:153:19: error: shift count >= width of type
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:33:94: note: expanded from macro 'BE32'
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:154:19: error: shift count >= width of type
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/hash_impl.h:33:32: note: expanded from macro 'BE32'
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen_impl.h:23:23: error: array is too large (1024 elements)
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/secp256.go:29:10: note: in file included from secp256.go!cgo.c:20:
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/secp256k1.c:16:10: note: in file included from /home/user/.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/./libsecp256k1/src/secp256k1.c:16:
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen_impl.h:33:43: error: array is too large (64 elements)
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen_impl.h:57:29: error: array is too large (1024 elements)
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen_impl.h:104:47: error: array is too large (64 elements)
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult.h:15:28: unknown C type: secp256k1_ge_storage[] (libclang type kind IncompleteArray)
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult.h:17:28: unknown C type: secp256k1_ge_storage[] (libclang type kind IncompleteArray)
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/secp256k1.c:50:30: expected a bitfield
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen.h:26:28: expected a bitfield
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen.h:27:22: expected a bitfield
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/ecmult_gen.h:28:19: expected a bitfield
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/secp256k1.c:51:34: expected a bitfield
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/secp256k1.c:52:24: expected a bitfield
../../../../.go/pkg/mod/github.com/ethereum/[email protected]/crypto/secp256k1/libsecp256k1/src/secp256k1.c:53:24: expected a bitfield

by any chance, are there cryptography libraries that support go-ethereum functionalities that I could plug in instead?

[dkegel-fastly]

I don't have an up to date dev tinygo at the moment, but you might try fiddling with build tags as Damian suggested.

The attached patch seemed to get me past the cgo problems and to needing IsZero: foo.patch

[irzhywau]

is there any appropriate way to workaround this issue? I seemingly have the same but with package runtime/metrics which doesn't exist in tinygo

[dkegel-fastly]

You could submit a stub for runtime/metrics to tinygo...?

Tinygo is making a fair bit of progress lately, but every little bit helps.

[lthibault]

@dkegel-fastly I might be interested in contributing runtime/metrics stubs. Where might I look to get started?

[dkegel-fastly]

Could be as simple as something like https://github.com/tinygo-org/tinygo/pull/1912 See https://tinygo.org/docs/guides/contributing/ for some tips, and ask on gophers.slack.com#tinygo-dev if you get stuck!