foundationdb-dotnet-client icon indicating copy to clipboard operation
foundationdb-dotnet-client copied to clipboard

Separate tuples into separate nuget package

Open slav opened this issue 5 years ago • 1 comments

FDB isn't the only kv store out there and your implementation to work with tuples for the key is applicable to any kv which accepts key as byte array. We use it not only to work with FDB but also to work with LMDB.

Would it be possible to publish tuples as an independent package? This way anyone working with any kv store or any other need to encode tuples into byte array will be able to benefit from your implementation without having to also pull FDB specific code and libraries.

slav avatar May 25 '19 16:05 slav

We are already doing something equivalent internally, with a custom build of the FoundationDB .NET Client that references a separate assembly that includes most of the Slice / Tuple / AsyncLINQ implementation that we use in our own applications.

Most of the code in the Shared sub-folder is conditionally included depending on the value of USE_SHARED_FRAMEWORK, which if undefined will use the local implementation or if defined will rely on these types being defined somewhere else.

So you could do your own version by extracting most of the code from the Shared folder to another assembly, create a new .csproj for the .NET Client that include all the source files from the fdb client project, and defines USE_SHARED_FRAMEWORK while at the same time referencing your external assembly.

This works well, but can be cumbersome because I have to periodically synchronize the code between both sides so that they are source-compatible.

Here is an example of that custom FoundationDB csproj that inludes files from a FoundationDB sub-module that is included in our app's repository.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net472</TargetFramework>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
    <LangVersion>latest</LangVersion>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <DefineConstants>$(DefineConstants);USE_SHARED_FRAMEWORK</DefineConstants>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <LangVersion>latest</LangVersion>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <DebugType>full</DebugType>
    <DebugSymbols>true</DebugSymbols>
    <DefineConstants>$(DefineConstants);USE_SHARED_FRAMEWORK</DefineConstants>
  </PropertyGroup>
  
  <ItemGroup>
    <ProjectReference Include="..\Doxense.Core\Doxense.Core.csproj" /> <!-- Contains Slice, Tuples, ... -->
    <ProjectReference Include="..\Doxense.Linq\Doxense.Linq.csproj" /> <!-- Contains Async LINQ -->
  </ItemGroup>
  
  <ItemGroup>
    <PackageReference Include="System.Memory" Version="4.5.1" />
    <PackageReference Include="System.Threading.Tasks.Extensions" Version="4.5.1" />
  </ItemGroup>

  <!-- Inject the source code from the .NET binding, EXCLUDING the "Shared" sub-folder (implemented in other assemblies) -->
  <ItemGroup>
    <Compile Include="..\..\FoundationDB\FoundationDB.Client\**\*.cs" />
  </ItemGroup>
  <ItemGroup>
    <Compile Remove="..\..\FoundationDB\FoundationDB.Client\Shared\**" />
  </ItemGroup>

</Project>

One advantage is that you can have a single "Core" assembly that has all your common and shared types that are used by most of your applications.

Regarding an "official" nuget package, it would be a possibility, but would mean that the Slice/Tuple API would need to be fixed and at the moment I have a few issues that may require a lot of changes.

This mainly boils down to the integration with Span<byte>/Memory<byte> for internal use, instead of the current Slice struct. The problem is that both Span and Memory are not exact replacements for the current Slice, and I'm also having performance issues when using Spans with something else than .NET Core 2.x or 3.0.

To be usable by most, this package would need to target .NET Standard 2.0, which does not know about Spans. On the other hand, .NET Core 3.0 (still in preview atm) has very fast span support and most of the API can accept spans directly... but I'm not sure if everyone has made the transition from .NET Framework to .NET Core yet...

I'd also would not like to have to maintain two .NET Framework-optimized and .NET Core-optimized versions in the interim.

My hope is that .NET Standard 2.1 will somehow solve the issue, and until it is available, I'm a bit unsure how to proceed.

KrzysFR avatar May 26 '19 13:05 KrzysFR