Json body response encoding of non-ascii characters

[federicocodo]

Assemblies affected At the moment I'm using the latest version of ASP.NET Core OData which has 8.0.10 as version.

Describe the bug I notice that using the Edm model of an entity and call it from the APIs it converts some characters into the Unicode character instead of representing them. In my example the oData API response convert the "§" into the Unicode character "\u00a7".

Reproduce steps

1. Download the following repository https://github.com/federicocodo/oDataTest
2. Start the WebApplication and call the WeatherForecast API http://localhost:5031/weatherforecast
3. In the json response you should see that in the "ToDecode" column there is always a "\u00a7" (this corresponds to the "§" character)

Data Model The Data model of the C# class can be found at the following link https://github.com/federicocodo/oDataTest/blob/master/WebApplicationTestoData/WeatherForecast.cs.

EDM (CSDL) Model This is the Edm model of the WeatherForecast C# class.

<edmx:Edmx xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx" Version="4.0">
    <edmx:DataServices>
        <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="WebApplicationTestoData">
            <EntityType Name="WeatherForecast">
                <Key>
                    <PropertyRef Name="Id"/>
                </Key>
                <Property Name="Id" Type="Edm.Guid" Nullable="false"/>
                <Property Name="Date" Type="Edm.DateTimeOffset" Nullable="false"/>
                <Property Name="TemperatureC" Type="Edm.Int32" Nullable="false"/>
                <Property Name="Summary" Type="Edm.String"/>
                <Property Name="ToDecode" Type="Edm.String" Nullable="false"/>
            </EntityType>
        </Schema>
        <Schema xmlns="http://docs.oasis-open.org/odata/ns/edm" Namespace="Default">
            <EntityContainer Name="Container">
                <EntitySet Name="WeatherForecast" EntityType="WebApplicationTestoData.WeatherForecast"/>
            </EntityContainer>
        </Schema>
    </edmx:DataServices>
</edmx:Edmx>

Request/Response The request used for the API call is the same of the previous one write in the steps to reproduce the issue: http://localhost:5031/weatherforecast. The response of the API call return this json:

{
  "@odata.context": "http://localhost:5031/$metadata#WeatherForecast",
  "value": [
    {
      "Id": "1620ef05-919b-4fb4-8413-f222f3829d5a",
      "Date": "2022-08-09T14:53:52.5736266+02:00",
      "TemperatureC": -13,
      "Summary": "Chilly",
      "ToDecode": "1119601080\u00a71702726918"
    },
    {
      "Id": "4a28017e-1865-4670-9c8b-aea6f482f97a",
      "Date": "2022-08-07T14:53:52.5736213+02:00",
      "TemperatureC": -3,
      "Summary": "Cool",
      "ToDecode": "1226886353\u00a7440318654"
    },
    {
      "Id": "82f3a8f1-887d-4781-828a-56d41564ccfa",
      "Date": "2022-08-06T14:53:52.5728782+02:00",
      "TemperatureC": 53,
      "Summary": "Balmy",
      "ToDecode": "834103851\u00a72146784071"
    },
    {
      "Id": "8af21294-5a0e-41a1-bd06-ac6dc086e8a4",
      "Date": "2022-08-08T14:53:52.5736261+02:00",
      "TemperatureC": 50,
      "Summary": "Bracing",
      "ToDecode": "577055983\u00a71635464599"
    },
    {
      "Id": "fc1b2f7d-2579-4479-b2f9-ddd565c79b15",
      "Date": "2022-08-10T14:53:52.573627+02:00",
      "TemperatureC": 17,
      "Summary": "Bracing",
      "ToDecode": "501313741\u00a7933620329"
    }
  ]
}

Expected behavior What I expect to see is that in the "ToDecode" column instead of reading "\u00a7" I should read the correct character it represents, from unicode, that is §.

Additional context During the various tests carried out on oData I noticed how removing the Edm model from the AddRouteComponents in the Program.cs causes the API response json to contain the "§" character not converted to Unicode ("\u00a7") in the "ToDecode" column. Here the json response:

[
  {
    "id": "31bd8c25-6c6e-453c-8bad-3289f6638988",
    "date": "2022-08-10T16:11:45.3894843+02:00",
    "temperatureC": 52,
    "temperatureF": 125,
    "summary": "Chilly",
    "toDecode": "1760004420§95151189"
  },
  {
    "id": "346158a9-a7ab-43f1-b4e5-e22d58c6cfa2",
    "date": "2022-08-09T16:11:45.389484+02:00",
    "temperatureC": 29,
    "temperatureF": 84,
    "summary": "Cool",
    "toDecode": "164081179§988381653"
  },
  {
    "id": "52315f11-8f0a-4efe-9ee3-faf0fc3e9ff1",
    "date": "2022-08-08T16:11:45.3894835+02:00",
    "temperatureC": -16,
    "temperatureF": 4,
    "summary": "Hot",
    "toDecode": "131708330§1616305195"
  },
  {
    "id": "cd19e3ee-e6e4-4938-9348-5ddeeb3ae7a5",
    "date": "2022-08-06T16:11:45.3889381+02:00",
    "temperatureC": 47,
    "temperatureF": 116,
    "summary": "Mild",
    "toDecode": "1634695148§1250621338"
  },
  {
    "id": "eada80b3-b1a4-4a09-94f5-4e4b7b5ebbe2",
    "date": "2022-08-07T16:11:45.3894805+02:00",
    "temperatureC": 48,
    "temperatureF": 118,
    "summary": "Warm",
    "toDecode": "1998032638§994706651"
  }
]

[xuzhg]

Just for your information: https://github.com/OData/odata.net/blob/master/src/Microsoft.OData.Core/Json/DefaultJsonWriterFactory.cs#L24

public DefaultJsonWriterFactory()
            : this(ODataStringEscapeOption.EscapeNonAscii)[4 years ago • Sam Xu [Fix issue #1181, refactor for the json string…]](https://sourcegraph.com/github.com/OData/odata.net/-/commit/6828bdfd85e12fd639a36b5d23b45ace9afdf444)
        {
        }

Need time to verify

[federicocodo]

After your suggestion I found in this issue #67 how to implement the DefaultJsonWriterFactory, is it correct? I tried to implement in my Program.cs but it doesn't seem to work, also I havn't found any documentation on how it is used after the dependency injection (if anything else needs to be done). You can find in my test repository how I am trying to implement the whole thing https://github.com/federicocodo/oDataTest/blob/master/WebApplicationTestoData/Program.cs.

[habbes]

@federicocodo if you want to override the default IJsonWriterFactory then you have to inject it in the core ODataWriter's service container, which is actually different from the AspNetCore's service container.

Here's how to inject a customized IJsonWriterFactory:

services.AddControllers()
             .AddOData(options =>
             options.AddRouteCompontents(“odata”, model, odataServices =>
               {
                       var jsonWriterFactory = new DefaultJsonWriterFactory(...);
                       services.AddSingleton<IJsonWriterFactory>(_ => jsonWriterFactory); 
                       odataServices.AddSingleton<IJsonWriterFactoryAsync>( => jsonWriterFactory);     
               });

Here's more information about DI support in OData's core library: https://docs.microsoft.com/en-us/odata/odatalib/di-support