SimpleS3 icon indicating copy to clipboard operation
SimpleS3 copied to clipboard

GetObject not working with NativeAOT

Open ivanjx opened this issue 2 years ago • 3 comments

  • [ ] I hereby verify that I am a sponsor.

Description of the bug PutObject works fine under NativeAOT but when i tried to get the file back with GetObject it gives me an empty stream instead. I manually verified that the PutObject works fine on both normal and AOT mode by downloading and checking the file myself.

How to reproduce?

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <PublishAot>true</PublishAot>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Genbox.SimpleS3.AmazonS3" Version="3.0.0" />
  </ItemGroup>

</Project>
using System.Text;
using Genbox.SimpleS3.AmazonS3;
using Genbox.SimpleS3.Core.Abstracts.Enums;
using Genbox.SimpleS3.Core.Common.Authentication;
using Genbox.SimpleS3.Core.Extensions;
using Genbox.SimpleS3.Core.Network.Responses.Objects;
using Genbox.SimpleS3.Extensions.AmazonS3;

AmazonS3Config config = new AmazonS3Config();
config.EndpointTemplate = "{Scheme}://play.min.io/{Bucket}";
config.Region = AmazonS3Region.UsEast1;
config.Credentials = new StringAccessKey("1nM2dy8VWatJ7EJ4nEBg", "CPeDvsHZA12kvcMXc6GJzdb6bIKo5FScCsYKKdgL");
config.NamingMode = NamingMode.PathStyle;

using AmazonS3Client client = new AmazonS3Client(config);
using MemoryStream dataStream = new MemoryStream(
    Encoding.ASCII.GetBytes("Hello World!! + " + DateTime.UtcNow.ToString("o")));
PutObjectResponse resp = await client.PutObjectAsync(
    "testbucket",
    "objectname22",
    dataStream);

GetObjectResponse resp2 = await client.GetObjectAsync("testbucket", "objectname22");
using StreamReader reader = new StreamReader(resp2.Content);
Console.WriteLine("Content: [{0}]", await reader.ReadToEndAsync());


Console.WriteLine("Success: " + resp.IsSuccess);

compile with: dotnet publish -c Release

normal output:

Content: [Hello World!! + 2023-08-05T04:00:19.6423971Z]
Success: True

NativeAOT output:

Content: []
Success: True

Expected behavior GetObject's Content has data in it.

ivanjx avatar Aug 05 '23 04:08 ivanjx

there is a workaround though by using the SignObject method:

...
string url = client.SignGetObject("testbucket", "objectname22", TimeSpan.FromMinutes(1));
Console.WriteLine("Url: {0}", url);
using HttpClient http = new HttpClient();
using Stream resp2Data = await http.GetStreamAsync(url);
using StreamReader reader2 = new StreamReader(resp2Data);
Console.WriteLine("Content Stream: [{0}]", await reader2.ReadToEndAsync());
...
Content: []
Url: https://play.min.io/testbucket/objectname22?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=1nM2dy8VWatJ7EJ4nEBg%2F20230805%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230805T041311Z&X-Amz-Expires=60&X-Amz-SignedHeaders=host&X-Amz-Signature=e1f575228ce0bcb0aae4d713bc620ad2ecc2f926f8221ed7955dc7382cd94298
Content Stream: [Hello World!! + 2023-08-05T04:13:10.2145266Z]
Success: True

ivanjx avatar Aug 05 '23 04:08 ivanjx

Interesting. I'll try and debug it once I get the time for it.

Genbox avatar Aug 05 '23 12:08 Genbox

I've taken a look at it now. There are several issues:

  1. RequestMarshal, ResponseMarshal, and PostMappers are matched via reflection. AOT removes all the unused types, so the marshallers are not registered.
  2. Profiles are serialized via JSON. AOT removes unused public constructors from types, so serialization does not work.
  3. Dependency injection uses dynamic code emitting. Currently, I'm using DI internally to ease the setup of clients.

Other potential problems:

  • Enums are handled via reflection and cached.
  • ObjectPool requires calling public ctors that gets removed

I've tried to root all assemblies, which should remove the trimming issues, but the application still crash - likely due to RequestMarshal being setup via DI.

So for the moment, NativeAOT is not supported by SimpleS3.

Genbox avatar Aug 13 '23 21:08 Genbox