Reduce executable size bloat (currently 20MB)

[awakecoding]

The current executable size is about 20MB on Windows, and slightly smaller on other platforms. While it is not a deal breaker, it wouldn't hurt to review the build process to see if the executable size could be reduced with compilation options.

• Look for common build options (optimize for size, trim debug symbols)
• Look at the list of dependencies and see which ones contribute to the bloat

[dopey]

via @awakecoding in gitter - https://medium.com/@jmrobles/gos-best-friend-upx-the-executable-compressor-e4f4872f1d8a

[dopey]

To follow up here - we don't have any immediate plans to work on reducing the size of the executable. However, the build process is all committed in Makefiles in this repo, so if anyone from the community would like to open a PR, we'd happily take the submission.

[Glandos]

I have some small tests results. First, sizes:

.rwxr-xr-x  31M user 2022-03-02 00:34 step-cli
.rwxr-xr-x  28M user 2022-03-04 09:48 step-cli.stripped
.rwxr-xr-x 6,4M user 2022-03-04 09:43 step-cli.stripped.upx
.rwxr-xr-x  12M user 2022-03-04 10:44 step-cli.stripped.upx1
.rwxr-xr-x 6,4M user 2022-03-04 09:43 step-cli.stripped.xz

The first on is the binary extracted from .deb. Second one is stripped with strip -s. Third one is upx --ultra-brute on the stripped version that took 10 minutes (yes, that's very long). Fourth one is upx -1 on the stripped version that took less than 2 seconds. Fifth one is xz -9 on the stripped version.

Now for launch timing:

/tmp ❯ hyperfine "./step-cli version"
Benchmark #1: ./step-cli version
  Time (mean ± σ):      18.3 ms ±   2.0 ms    [User: 15.6 ms, System: 9.0 ms]
  Range (min … max):    14.8 ms …  25.3 ms    135 runs
 
/tmp ❯ hyperfine "./step-cli.stripped version"
Benchmark #1: ./step-cli.stripped version
  Time (mean ± σ):      18.3 ms ±   1.9 ms    [User: 16.3 ms, System: 8.5 ms]
  Range (min … max):    14.4 ms …  24.0 ms    160 runs
 
/tmp ❯ hyperfine "./step-cli.stripped.upx version"
Benchmark #1: ./step-cli.stripped.upx version
  Time (mean ± σ):     523.7 ms ±  51.3 ms    [User: 519.2 ms, System: 8.0 ms]
  Range (min … max):   446.8 ms … 645.9 ms    10 runs

/tmp ❯ hyperfine "./step-cli.stripped.upx1 version" --warmup 1
Benchmark #1: ./step-cli.stripped.upx1 version
  Time (mean ± σ):     127.5 ms ±   7.9 ms    [User: 125.3 ms, System: 6.7 ms]
  Range (min … max):   121.2 ms … 147.7 ms    24 runs

So the result is that if RPM and DEB are compressed with XZ, this is fine. The size on disk will be a little bit bigger, but it's usually harmless. I've noticed that the current DEB version is using gzip, that should be changed. You should at least strip binary to have a quick 10% on size. But then, debugging is harder.

UPX can be used, but it really slows down binary start.

[dopey]

Hey, thanks for posting this @Glandos! It's great to have some concrete numbers.

So we are aware that the executable size is relatively large for cli. However, that comes with the territory - statically linked golang binaries. We could strip it down but we'd be losing the debugging facilities you mentioned, and we'd prefer not to do that - at least not by default. We'd be happy to have a target added to the makefile that could build a reduced size executable.

Furthermore, we'd like to understand the case better (beyond just smaller is better).

• Is this for a constrained device? If so then how much space are we actually working with?
• Should we be considering removing functional dependencies for facets of the cli that are uncommonly used. For example, if you're not planning on using ssh we could remove those libraries, or libraries for OTT, etc.