NuGet package size concerns

[0xced]

The current NuGet package (version 3.1.3) weights 115 MB.

Commit a932934998c298f0ea1386cc7b9cfbf7fdcc6f10 added support for Linux/arm64. Creating a new NuGet package with MongoDB 5.1.1 and the database tools 100.5.1 now weights 146 MB. It would weight 180 MB without stripping binaries (introduced in 67dd1e9f4dcf9797ab941bed6b26d9bc1d6a2e09).

I think this starts to become problematic. 146 MB is a lot, even by todays standards. Also, 4 different platforms/architectures are provided by the package but only one will be usable. For example, the x64 ELF mongod binary will only run on Linux, it is a waste to download it on macOS and Windows.

I can think of two solutions to this problem.

1. Split the Mongo2Go package like SkiaSharp is doing with SkiaSharp.NativeAssets.Linux, SkiaSharp.NativeAssets.macOS and SkiaSharp.NativeAssets.Win32. We could keep the core Mongo2Go package for the core functionality and provide 4 packages for the native assets (i.e. binary executables): Mongo2Go.Native.Linux.arm64, Mongo2Go.Native.Linux.x64, Mongo2Go.Native.macOS.arm64 and Mongo2Go.Native.Windows.x64.

2. Do not provide the binary executable as part of the NuGet package but download them on demand instead.

My preference goes to the second option because it solves the package size issue and it even enables the consumer to choose the MongoDB version it wants at runtime. This would also untie the MongoDB version from the Mongo2Go version. The first option would still require to produce new NuGet package for new MongoDB versions whereas the second option completely solves this issue.

I have already started working on a branch (split-downloader-cli_parts) to re-use the code that was written for the MongoDownloader tool so that it becomes usable directly from Mongo2Go. The idea is for the consumer to be able to specify which version of MongoDB it wants using the familiar NuGet version range syntax. I have already implemented the best version match in my branch (leveraging the NuGet.Versioning package).

[JohannesHoppe]

👍 👍

Yes, the second version sounds like a good plan. But here we will have massive problems with proxies. In my job I do workshops for companies and it is always hell to get a simple download via NPM.

If we go this way, we absolutely have to consider environment variables with HTTP_PROXY, HTTPS_PROXY, NO_PROXY, if possible the Windows proxy settings and to be on the safe side also proxy settings by code.

[JohannesHoppe]

But on the other hand, we would no longer have any trouble finding the right folder. That would make the whole complicated search logic obsolete. I am very excited.

[JohannesHoppe]

According to you code you are going to make a CLI. That looks good. Are you also planning a mode where you just call Mongo2Go as usual (without a CLI) and it just downloads the latest version without asking?

[0xced]

Yes, the second version sounds like a good plan. But here we will have massive problems with proxies. In my job I do workshops for companies and it is always hell to get a simple download via NPM.

I get you, I have spent waaaaaay too much time and energy resolving proxy issues on company networks. I even had to write tools to diagnose proxy issues! 😅

If we go this way, we absolutely have to consider environment variables with HTTP_PROXY, HTTPS_PROXY, NO_PROXY, if possible the Windows proxy settings and to be on the safe side also proxy settings by code.

Absolutely ! This is already planned actually. A default HttpClient instance is provided but could be replaced by a custom one. Note that HttpClient observes the HTTP_PROXY, HTTPS_PROXY, ALL_PROXY and NO_PROXY environment variables and the system settings by default so this should be seldom needed.

[0xced]

According to you code you are going to make a CLI. That looks good. Are you also planning a mode where you just call Mongo2Go as usual (without a CLI) and it just downloads the latest version without asking?

Yes absolutely. I just split the original MongoDownloader project which was the CLI for automatically downloading the MongoDB binaries into MongoDownloader (core downloader classes) + MongoDownloader.CLI (command line interface to download the packages). I split into two different projects to make sure the CLI-specific parts (Spectre.Console) are kept in the CLI tool only.

My plan is to eventually move all the classes from MongoDownloader into Mongo2Go and automatically use the downloader from there, automatically handling where to download the binaries.

This will inevitably be a breaking change. I think we should choose a sensible default for where to download the executable binaries that the user could override. I was thinking ~/Library/Application Support/Mongo2Go for macOS, %LocalAppData%\Mongo2Go on Windows and ~/.local/share/Mongo2Go for Linux.

[JohannesHoppe]

This will inevitably be a breaking change.

Version bump to v4 for sure!

One last v3 to address #131 and #132 and to use the current downloader for the first and last time! 😇

[0xced]

Dropping a quick note here: it seems it could be possible to package the native binaries into their own separate NuGet packages and have NuGet automatically restore the native package matching the current runtime. At least it's my understanding of what should happen when looking at the Microsoft.NETCore.DotNetAppHost package. It only has a runtime.json file that lists native dependencies, no lib directory, no runtimes directory at all! I could not find documentation about this kind of NuGet package with only a runtime.json file, but maybe I just did not research well enough.

{
  "runtimes": {
    "freebsd-x64": {
      "#import": [],
      "Microsoft.NETCore.DotNetAppHost": {
        "runtime.freebsd-x64.Microsoft.NETCore.DotNetAppHost": "[6.0.3, )"
      }
    },
    "linux-arm": {
      "#import": [],
      "Microsoft.NETCore.DotNetAppHost": {
        "runtime.linux-arm.Microsoft.NETCore.DotNetAppHost": "[6.0.3, )"
      }
    },
    "linux-arm64": {
      "#import": [],
      "Microsoft.NETCore.DotNetAppHost": {
        "runtime.linux-arm64.Microsoft.NETCore.DotNetAppHost": "[6.0.3, )"
      }
    },
…

I still think it's preferable to have the Mongo2Go main package to automatically download the mongo binaries but I just wanted to mention this option which could also help reducing the size of the main package.

[asimmon]

I've read a lot about runtime.json, I summarized my findings in https://github.com/asimmon/ephemeral-mongo/issues/2 . TLDR: it is not a viable option.

I still wanted to do something about it while supporting new MongoDB versions, that's why I created https://github.com/asimmon/ephemeral-mongo