JSON deserialization fails in iOS release build if class implements an interface

[markuspalme]

Description

A simple JSON deserialization using Newtonsoft.Json fails on iOS in a release build as soon as the class that is trying be be de-serialized implements an interface.

The same app works in Debug build in the iOS simulator as expected.

Reproduction Steps

1. Create a new iOS app with dotnet new ios

2. Set a valid bundle identifier that allows installing the app on a physical device

3. Add this code to the project:

   public class ProductImage : IEntity
   {
      public string ProductNumber { get; set; }
      public Guid Id { get; set; }
   }
   
   public interface IEntity
   {
      Guid Id { get; set; }
   }
   

   During startup, run this code:

   var x = @"{
      ""productNumber"": ""P1"",
      ""id"": ""46c67d7c-fd15-4d89-9fed-991c48c1bacf""
   }";
   
   var pi = JsonConvert.DeserializeObject<ProductImage>(x);
   
   

4. Build the app for a device: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained

5. Deploy to device and run

Full project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-ios</TargetFramework>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <SupportedOSPlatformVersion>13.0</SupportedOSPlatformVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
</Project>

Self-contained example project: https://github.com/markuspalme/dotnetruntime-75802

Expected behavior

The app starts successfully and manages to de-serialize the JSON regardless of whether the class implements an interface or not.

Note that removing the interface will make the de-serialization work as expected.

Actual behavior

JSON deserialization fails with this exception:

Error setting value to 'Id' on 'testapp.ProductImage'.

Regression?

The same code works in Xamarin.iOS.

Known Workarounds

No response

Configuration

.NET 6.0.401 iOS 16, same on 15.6.1

Other information

No response

[ghost]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

A simple JSON deserialization using Newtonsoft.Json fails on iOS in a release build as soon as the class that is trying be be de-serialized implements an interface.

The same app works in Debug build in the iOS simulator as expected.

Reproduction Steps

1. Create a new iOS app with dotnet new ios
2. Set a valid bundle identifier that allows installing the app on a physical device
3. Add this code to the project:

  public class ProductImage : IEntity
  {
	  public string ProductNumber { get; set; }
  
	  public Guid Id { get; set; }
  }
  
  public interface IEntity
  {
	  Guid Id { get; set; }
  }
 

During startup, run this code:

var x = @"{
      ""productNumber"": ""P1"",
      ""id"": ""46c67d7c-fd15-4d89-9fed-991c48c1bacf""
  }";

var pi = JsonConvert.DeserializeObject<ProductImage>(x);

4. Build the app for a device: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained

5. Deploy to device and run

Full project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-ios</TargetFramework>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <SupportedOSPlatformVersion>13.0</SupportedOSPlatformVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
</Project>

Expected behavior

The app starts successfully and manages to de-serialize the JSON regardless of whether the class implements an interface or not.

Note that removing the interface

Actual behavior

JSON deserialization fails with this exception:

Error setting value to 'Id' on 'testapp.ProductImage'.

Regression?

The same code works in Xamarin.iOS.

Known Workarounds

No response

Configuration

.NET 6.0.4.1 iOS 16

Other information

No response

Author:	markuspalme
Assignees:	-
Labels:	area-System.Text.Json, untriaged
Milestone:	-

[gregsdennis]

Are you certain this is a bug in .Net? Seems like it's a Newtonsoft issue. If so, the bug should be filed in that repo.

[markuspalme]

@gregsdennis It works just fine in net6 on Windows, MacOS and also in the iOS simulator - all with the same Newtonsoft.Json package. So I think this is a runtime or maybe AOT issue (iOS requires AOT).

[gregsdennis]

Understood, but it might be that the package just needs handle a special case in this OS. There is the possibility that the issue is the package.

[markuspalme]

Unlikely, the same code works with Xamarin.iOS.

[Cheesebaron]

@markuspalme could it be the linker stripping out stuff?

Could you try add <PublishTrimmed>false</PublishTrimmed> in your Project Properties and see if that helps?

[markuspalme]

@Cheesebaron Thanks for the suggestion, on iOS that is not an option though:

[ghost]

Tagging subscribers to 'os-ios': @steveisok, @akoeplinger See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

A simple JSON deserialization using Newtonsoft.Json fails on iOS in a release build as soon as the class that is trying be be de-serialized implements an interface.

The same app works in Debug build in the iOS simulator as expected.

Reproduction Steps

1. Create a new iOS app with dotnet new ios
2. Set a valid bundle identifier that allows installing the app on a physical device
3. Add this code to the project:

  public class ProductImage : IEntity
  {
	  public string ProductNumber { get; set; }
  
	  public Guid Id { get; set; }
  }
  
  public interface IEntity
  {
	  Guid Id { get; set; }
  }
 

During startup, run this code:

var x = @"{
      ""productNumber"": ""P1"",
      ""id"": ""46c67d7c-fd15-4d89-9fed-991c48c1bacf""
  }";

var pi = JsonConvert.DeserializeObject<ProductImage>(x);

4. Build the app for a device: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained

5. Deploy to device and run

Full project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-ios</TargetFramework>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <SupportedOSPlatformVersion>13.0</SupportedOSPlatformVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
</Project>

Expected behavior

The app starts successfully and manages to de-serialize the JSON regardless of whether the class implements an interface or not.

Note that removing the interface will make the de-serialization work as expected.

Actual behavior

JSON deserialization fails with this exception:

Error setting value to 'Id' on 'testapp.ProductImage'.

Regression?

The same code works in Xamarin.iOS.

Known Workarounds

No response

Configuration

.NET 6.0.401 iOS 16, same on 15.6.1

Other information

No response

Author:	markuspalme
Assignees:	-
Labels:	area-System.Text.Json, untriaged, os-ios
Milestone:	-

[markuspalme]

I have tried System.Text.Json instead of Newtonsoft.Json. It does not throw an exception, but it does not set any property values.

var pi = System.Text.Json.JsonSerializer.Deserialize<ProductImage>(x);

[markuspalme]

Here's a small self-contained example: https://github.com/markuspalme/dotnetruntime-75802

[markuspalme]

Adding <UseInterpreter>true</UseInterpreter> as suggested in this issue helps - but I don't think this should be neccesary: https://github.com/xamarin/xamarin-macios/issues/15961

[Cheesebaron]

@markuspalme UseInterpreter is not for Release builds, Apple won't allow that.

[markuspalme]

@Cheesebaron Yes, I figured but wanted to share that it helps here - maybe it helps pinpointing the root cause.

[ghost]

Tagging subscribers to this area: @dotnet/area-system-text-json, @gregsdennis See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

A simple JSON deserialization using Newtonsoft.Json fails on iOS in a release build as soon as the class that is trying be be de-serialized implements an interface.

The same app works in Debug build in the iOS simulator as expected.

Reproduction Steps

1. Create a new iOS app with dotnet new ios

2. Set a valid bundle identifier that allows installing the app on a physical device

3. Add this code to the project:

   public class ProductImage : IEntity
   {
      public string ProductNumber { get; set; }
      public Guid Id { get; set; }
   }
   
   public interface IEntity
   {
      Guid Id { get; set; }
   }
   

   During startup, run this code:

   var x = @"{
      ""productNumber"": ""P1"",
      ""id"": ""46c67d7c-fd15-4d89-9fed-991c48c1bacf""
   }";
   
   var pi = JsonConvert.DeserializeObject<ProductImage>(x);
   
   

4. Build the app for a device: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained

5. Deploy to device and run

Full project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-ios</TargetFramework>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <SupportedOSPlatformVersion>13.0</SupportedOSPlatformVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
</Project>

Self-contained example project: https://github.com/markuspalme/dotnetruntime-75802

Expected behavior

The app starts successfully and manages to de-serialize the JSON regardless of whether the class implements an interface or not.

Note that removing the interface will make the de-serialization work as expected.

Actual behavior

JSON deserialization fails with this exception:

Error setting value to 'Id' on 'testapp.ProductImage'.

Regression?

The same code works in Xamarin.iOS.

Known Workarounds

No response

Configuration

.NET 6.0.401 iOS 16, same on 15.6.1

Other information

No response

Author:	markuspalme
Assignees:	-
Labels:	area-System.Text.Json, untriaged, os-ios
Milestone:	-

[eiriktsarpalis]

@jeffschwMSFT I don't believe the issue is related to System.Text.Json. I removed the label but wasn't sure what the appropriate area label should be. Perhaps @steveisok or @akoeplinger know.

[jkotas]

System.Text.Json should be the right area label.

Source generators are the only 100% reliable way to perform Json serialization/deserialization in the presence of trimming or AOT compilation without fallbacks.

Unfortunately, trim warnings that would notify about this problem are disabled for iOS apps by default. You can set <SuppressTrimAnalysisWarnings>false</SuppressTrimAnalysisWarnings> in your .csproj to enable trim warnings to see all potential places in your app that can break due to trimming.

This is duplicate https://github.com/dotnet/runtime/issues/74141 and number of other similar issues.

@eiriktsarpalis My suggestion would be:

• Make sure that we have documentation that we can point to and resolve these issues against. Do we have a documentation that says "System.Text.Json source generators are the only 100% reliable way to perform Json serialization/deserialization in the presence of trimming"?
• Note the feedback on issues that track enabling the link warning in app-model specific SDKs. It is https://github.com/xamarin/xamarin-macios/issues/11246 for iOS.

[eiriktsarpalis]

System.Text.Json should be the right area label.

The OP concerns Json.NET (Newtonsoft) serialization issues in iOS. I agree that it falls under the same category as #74141, but I wonder what our approach should be when users report trimming-related failures when using third party reflection-based libraries. We might want to consider asking them to file an issue with said third-party libraries asking about AOT support.

Do we have a documentation that says "System.Text.Json source generators are the only 100% reliable way to perform Json serialization/deserialization in the presence of trimming"?

It is, although the wording is certainly more understated than that.

[markuspalme]

@jkotas I will try to enable the trim warnings.

JSON is at the heart of many apps and this feels like a regression coming from Xamarin.iOS where such scenarios worked out of the box. Migrating toSystem.Text.Json source generators is a possibility for user code, but many libraries depend on Newtonsoft.Json, consider a generic key-value store like Akavache (https://github.com/reactiveui/Akavache)

[eiriktsarpalis]

@markuspalme I'm not sure if and when Json.NET plans on bringing AOT support, but you might want to consider opening an issue in its own repo or those of the other libraries that depend on it. In the meantime, migrating to System.Text.Json source generation is probably the only viable option.

[markuspalme]

@eiriktsarpalis Thanks for your input. Given that earlier versions of Xamarin.iOS supported this scenario out of the box even in AOT mode, this seems like a bad surprise or even a blocker for anyone porting applications to net6-ios.

[markuspalme]

@eiriktsarpalis So my understanding is that this problem here is caused by trimming. To preserve all the types in my code, I have added this to the project file as described here (https://devblogs.microsoft.com/dotnet/customizing-trimming-in-net-core-5/):

<ItemGroup>
  <TrimmerRootDescriptor Include="TrimmerRoots.xml" />
</ItemGroup>

The TrimmerRoots.xml looks like this:

<?xml version="1.0" encoding="utf-8"?>

<linker>
    <assembly fullname="testapp" preserve="all" />
</linker>

The de-serialization does not work with that in place. Shouldn't this ensure that all my types are preserved and available through reflection?

The whole story around trimming and linking is rather intimidating:

• there is the [Preserve] attribute
• there is the before mentioned external XML configuration option
• For net6-ios projects there is the MtouchLink property in the project file

Is there any guidance available on which one to use when and how they work together?

[ghost]

Tagging subscribers to 'linkable-framework': @eerhardt, @vitek-karas, @LakshanF, @sbomer, @joperezr See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

A simple JSON deserialization using Newtonsoft.Json fails on iOS in a release build as soon as the class that is trying be be de-serialized implements an interface.

The same app works in Debug build in the iOS simulator as expected.

Reproduction Steps

1. Create a new iOS app with dotnet new ios

2. Set a valid bundle identifier that allows installing the app on a physical device

3. Add this code to the project:

   public class ProductImage : IEntity
   {
      public string ProductNumber { get; set; }
      public Guid Id { get; set; }
   }
   
   public interface IEntity
   {
      Guid Id { get; set; }
   }
   

   During startup, run this code:

   var x = @"{
      ""productNumber"": ""P1"",
      ""id"": ""46c67d7c-fd15-4d89-9fed-991c48c1bacf""
   }";
   
   var pi = JsonConvert.DeserializeObject<ProductImage>(x);
   
   

4. Build the app for a device: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained

5. Deploy to device and run

Full project file:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-ios</TargetFramework>
    <OutputType>Exe</OutputType>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <SupportedOSPlatformVersion>13.0</SupportedOSPlatformVersion>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
  </ItemGroup>
</Project>

Self-contained example project: https://github.com/markuspalme/dotnetruntime-75802

Expected behavior

The app starts successfully and manages to de-serialize the JSON regardless of whether the class implements an interface or not.

Note that removing the interface will make the de-serialization work as expected.

Actual behavior

JSON deserialization fails with this exception:

Error setting value to 'Id' on 'testapp.ProductImage'.

Regression?

The same code works in Xamarin.iOS.

Known Workarounds

No response

Configuration

.NET 6.0.401 iOS 16, same on 15.6.1

Other information

No response

Author:	markuspalme
Assignees:	-
Labels:	question, area-System.Text.Json, linkable-framework, os-ios, trimming-for-aot
Milestone:	-

[vitek-karas]

The trimming options are described here: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trimming-options?pivots=dotnet-6-0

The Preserve attribute has been deprecated... I don't actually think it works anymore (but maybe it does for iOS targets)

The trimmer should produce warnings if the XML is not understood - but mobile targets disable these warnings by default (there are just way too many warnings produced by all layers of the system currently). You can turn it on by setting SuppressTrimAnalysisWarnings=false and look for warnings pointing to your XML file.

@sbomer for additional details.

[markuspalme]

@vitek-karas I did that and get the expected warning about Newtonsoft.Json. But all types that are involded in the de-serialization are marked for preservation in the XML file.

There are no warnings regarding the XML file. You can access the full project here: https://github.com/markuspalme/dotnetruntime-75802

[vitek-karas]

@sbomer - could you please try to take a look if the XML descriptor works as expected in this case?

[sbomer]

I tried the repro as a win-x64 console app and the descriptor you shared fixes deserialization, so this might be specific to the Xamarin SDK. I am not currently set up to publish this for iOS. @markuspalme would you be able to share a binlog of the publish command?

[markuspalme]

@sbomer here you go: build.binlog.zip

Build command: dotnet publish testapp.csproj -f:net6.0-ios -c:Release /p:CodesignKey="Apple Development: ******" /p:CodesignProvision="Test app" /p:ArchiveOnBuild=true -r ios-arm64 --self-contained -bl:build.binlog

[markuspalme]

@sbomer Did the binlog reveal anything interesting?

[steveisok]

@sbomer if you need to bounce ideas off of someone who has ios set up, please let me know.

[sbomer]

Sorry for the delay - I wasn't able to see anything wrong from the binlog, which indicates that the descriptor is getting passed along correctly. @vitek-karas would your repro tool help here to collect the input assemblies?