Binary size, WE CAN GO FURTHERRRRR

[Jeffail]

So we recently made a large effort to pull out most component implementations into optional imports, that was a right bother but we ain't not never benefited none because you can get a way smaller binary now by compiling with a more selective set of components.

However, running the very cool https://github.com/jondot/goweight on ./cmd/benthos with all the optional imports omitted we get this at the top:

  9.8 MB runtime
  7.9 MB net/http
  6.8 MB github.com/benthosdev/benthos/v4/internal/bloblang/query
  3.9 MB golang.org/x/crypto/ssh
  3.9 MB net
  3.6 MB github.com/xeipuuv/gojsonschema
  3.6 MB go.mongodb.org/mongo-driver/mongo
  3.5 MB crypto/tls
  3.4 MB reflect
  3.3 MB github.com/pkg/sftp
  2.9 MB github.com/urfave/cli/v2
  2.7 MB github.com/klauspost/compress/zstd
  2.6 MB go.mongodb.org/mongo-driver/bson/bsoncodec
  2.4 MB go.mongodb.org/mongo-driver/x/mongo/driver/topology
  2.3 MB go.mongodb.org/mongo-driver/x/mongo/driver/operation
  2.1 MB math/big
  2.1 MB gopkg.in/yaml.v3
  2.0 MB github.com/rcrowley/go-metrics
  1.9 MB go.mongodb.org/mongo-driver/x/bsonx
  1.9 MB go.mongodb.org/mongo-driver/bson/bsonrw

Which means we still get quite a few packages that are clearly not required in a benthos thats void of components, such as all the MongoDB packages listed, sftp, gojsonschema, etc.

This is, in technical terms, whack. We should spend a bit more time pushing these imports out of default builds and then report back on what our final minimum binary size is.

[mihaitodor]

Very cool! I have a MongoDB refactor to switch it to the new public API in the works, but it’s been marinating on a branch for some weeks now 😔 I’ll resurrect it ASAP.