lightning-node-connect
lightning-node-connect copied to clipboard
lightninglabs
build: investigate shrinking WASM blob size using tiny-go
Was brought up in this twitter thread: https://twitter.com/roasbeef/status/1486422025816064000
TinyGo is an alternate Go compiler that is intended for embedded systems, and support compiling to a WASM target: https://tinygo.org/docs/reference/usage/subcommands/#build.
Their overview page demonstrators shrinking a binary size by 99% just by instead compiling using the Tiny Go compiler. Certain Go language features aren't fully supported, so we may need to add some additional build tag layers to make it possible. So far, someone has identified that we'll need to build tag out the pprof import at the very least.
runtime/trace package is incompatible. This breaks all the OpenTelemetry stuff https://github.com/tinygo-org/tinygo/issues/1997
stdlib compatibility list here: https://tinygo.org/docs/reference/lang-support/stdlib/
I've assessed all the dependencies in LNC directly, and pulled in through vendor
issues with imports:
archive/tar crypto/ed25519 crypto/tls crypto/x509 encoding/xml expvar net/http net/http/httptest net/http/httptrace net/http/httputil net/http/pprof os/user
issues with tests:
bufio bytes compress/gzip context crypto crypto/aes crypto/cipher crypto/ecdsa crypto/elliptic crypto/hmac crypto/rsa crypto/subtle database/sql database/sql/driver encoding/asn1 encoding/base64 encoding/binary encoding/json encoding/pem errors flag fmt go/token hash/crc32 html/template image/color io io/ioutil log log/syslog math/big math/bits math/rand mime net/textproto net/url os/exec os/signal path/filepath regexp sort strconv strings sync/atomic syscall text/tabwriter
Re OpenTelemetry, looks like it's just a dep brought in thru aperture:
⛰ go mod why go.opentelemetry.io/contrib
# go.opentelemetry.io/contrib
github.com/lightninglabs/lightning-node-connect/itest
github.com/lightninglabs/aperture
github.com/lightninglabs/aperture.test
go.etcd.io/etcd/server/v3/embed
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
go.opentelemetry.io/contrib
In this case, it's only really required atm for the itests, so there might just need to add a build tag workaround.
Not being able to import net/http seems like a pretty fundamental barrier.
However, I wonder if it's possible for us to take a multi-pronged approach here. With this hypothetical path, we'd actually have multiple wasm files, with us trying to pack as much into the tinygo compiled file as possible. I'm not sure if this is actually possible based on the way the project is architected however.
In terms of shrinking things after the fact, looks like we might be able to use this wasm-opt tool to strip things down a bit more (namely the debug information): https://github.com/WebAssembly/binaryen#binaryen-optimizations.
Here's a blog post that shows an example near the end: https://www.fermyon.com/blog/optimizing-tinygo-wasm#Reducing%20More%20with%20Wasm-opt
I tried gzip'ing locally, and got a pretty nice size reduction:
-rwxr-xr-x 1 roasbeef staff 42M Jul 14 15:32 wasm-client.wasm
-rwxr-xr-x 1 roasbeef staff 8.9M Jul 14 15:32 wasm-client.wasm.gz
Are there any updates on this? I think it would be great to make more optimizations for file size, the current WASM file is sitting at 57.1mb. I was able to get it down to 49.6mb using wasm-opt which is an improvement but still not great.
