ephemeral-mongo icon indicating copy to clipboard operation
ephemeral-mongo copied to clipboard

Optimize NuGet package downloads for a specific operating system

Open asimmon opened this issue 2 years ago • 3 comments

EphemeralMongo use a dedicated runtime NuGet package for each operating system (linux-x64, osx-x64 and windows-x64).

However, when using EphemeralMongo4, EphemeralMongo5 or EphemeralMongo6, the three OS-specific runtime packages are downloaded and copied to the build output, which takes time and a lot of space. The ideal solution would to only download the relevant runtime package for the consuming operating system.

Unfortunately, there is no viable solution today to optimize the download process.

  1. It is not an option to download MongoDB and the tools at runtime, there could be network issues, some people use proxies and firewalls, we would have to implement a reliable cache.
  2. .NET does not provide (yet) a way to use conditional NuGet dependencies on both the target framework moniker (TFM) and a runtime identifier (RID).

There are actually a lot of opened issues in the .NET GitHub repository about option #2, so it might be possible in a future version of .NET:

  • https://github.com/NuGet/Home/issues/1660
  • https://github.com/NuGet/Home/issues/5862
  • https://github.com/NuGet/Home/issues/10571
  • https://github.com/dotnet/runtime/issues/49137
  • https://github.com/dotnet/sdk/issues/4552

Using runtime.json is also not an option because it is deprecated, undocumented, and was created for internal .NET Core NuGet packages.

Until then, we'll stick with the RID-specific runtime package.

asimmon avatar Jul 31 '22 17:07 asimmon

I finally got around to try EphemeralMongo. 😃

It seems to me that there's an easy solution to reduce the download size with the existing packages. Instead of importing EphemeralMongo through the EphemeralMongo6 package, one can import the EphemeralMongo.Core package and the runtime package conditionally, so that only the runtime for the current OS is restored (i.e. downloaded).

In other technical words, going from this:

<ItemGroup>
  <PackageReference Include="EphemeralMongo6" Version="0.1.3" />
</ItemGroup>

to this:

<ItemGroup>
  <PackageReference Include="EphemeralMongo.Core" Version="0.1.3" />
  <PackageReference Include="EphemeralMongo6.runtime.linux-x64" Version="0.1.3" Condition="$([MSBuild]::IsOSPlatform('Linux'))" />
  <PackageReference Include="EphemeralMongo6.runtime.osx-x64" Version="0.1.3" Condition="$([MSBuild]::IsOSPlatform('OSX'))" />
  <PackageReference Include="EphemeralMongo6.runtime.win-x64" Version="0.1.3" Condition="$([MSBuild]::IsOSPlatform('Windows'))" />
</ItemGroup>

It's not as tidy and a built-in solution would be nicer but it can definitely save some bandwidth and time.

Maybe it could be worth mentioning this approach in the README?

0xced avatar Feb 04 '23 00:02 0xced

@0xced done! Thank you for the suggestion.

image

asimmon avatar Feb 22 '23 18:02 asimmon

Linking some more issues that discuss the matter (even if not obvious from their titles).

  • https://github.com/dotnet/core/issues/7568
  • https://github.com/NuGet/NuGetGallery/issues/9473
  • https://github.com/dotnet/sdk/issues/33845

Interesting how all theses issues are still open. 😅

Also, note that the libclang package is using the undocumented runtime.json feature.

0xced avatar Sep 30 '23 22:09 0xced