Explosion in code source size

[Cimbali]

Describe the bug Looking at tarballs of the last 15ish releases, the code size has increased almost 10-fold:

29M     mozillavpn_2.11.0.orig.tar.gz
64M     mozillavpn_2.12.0.orig.tar.gz
64M     mozillavpn_2.13.0.orig.tar.gz
66M     mozillavpn_2.13.1.orig.tar.gz
67M     mozillavpn_2.14.0.orig.tar.gz
67M     mozillavpn_2.14.1.orig.tar.gz
93M     mozillavpn_2.15.0.orig.tar.gz
93M     mozillavpn_2.15.3.orig.tar.gz
100M    mozillavpn_2.16.1.orig.tar.gz
101M    mozillavpn_2.17.0.orig.tar.gz
114M    mozillavpn_2.18.0.orig.tar.gz
114M    mozillavpn_2.18.1.orig.tar.gz
115M    mozillavpn_2.19.0.orig.tar.gz
116M    mozillavpn_2.20.0.orig.tar.gz
117M    mozillavpn_2.21.0.orig.tar.gz
103M    mozillavpn_2.22.0.orig.tar.gz
216M    mozillavpn_2.23.1.orig.tar.gz
222M    mozillavpn_2.24.0.orig.tar.gz

VPN version: 2.24.0

Affected Platforms: Linux

Prerequisites: None

Steps to reproduce: None

I’m not sure why this is happening, but the codebase doesn’t seem to be actually getting that much larger. There’s a gigantic jump at version 2.23 (≥110M increase).

One thing is that it seems that more and more a lot of various related codebases get vendored into linux source bundles (inluding android and ios sdks? windows msvc go packages?). Would it be possible to try to have a look at this and try to limit this effect? It’s not like you’re supporting a ton of Linux platforms anyway (I might have missed something, but the download page still says “Ubuntu 20.04 and up”).

[oskirby]

I think the explosion in size has been largely caused by some mismanagement of our rust crates. A PR landed in version 2.23 (https://github.com/mozilla-mobile/mozilla-vpn-client/pull/9585) which attempted to unify our various Cargo projects into one workspace but this wound up causing each crate to duplicate the vendored crates in their own subdirectory.

This got cleaned up in my last round of cleanup work for the Flatpak support in PR #9798 where I unified all of the cargo vendor directories into one. After this, the source tarball is back to something on the order of 100MB as built from the main branch. So we hope that the 2.25 release will shrink back down in size.

Most of this explosion in the source code size is due to the vendored crates required by the 3rdparty/glean module, which contributes approximately 50MB of compressed data to the archive size (on 2.23 and 2.24 this 50MB was duplicated a couple of times over).

We have also seen the size of the sources grow by a couple of megabytes due to the addition of the 3rdparty/sentry module which we use for crash reporting. This size of this submodule was initially pretty small, but it grew rather significantly to about 7.5MB of compressed data as we implemented this on more platforms

[oskirby]

There is some inefficiency in the vendored rust crates too, which is caused by some version mismatches. For example, we wind up with three versions of windows-sys due to various version pins.