AspNetCoreOData
AspNetCoreOData copied to clipboard
Published
20 hours ago •
OData
OData
JSON Serialization Issue with OData and Self-Referencing Complex Types
Assemblies affected
Microsoft.AspNetCore.OData 8.2.3
Microsoft.Azure.Cosmos 3.35.4
Framework dotnet 6
Dependencies
<PackageReference Include="Microsoft.AspNetCore.Mvc.NewtonsoftJson" Version="6.0.22" />
<PackageReference Include="Microsoft.AspNetCore.OData" Version="8.2.3" />
<PackageReference Include="Microsoft.AspNetCore.OData.NewtonsoftJson" Version="8.2.0" />
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="7.0.5"/>
<PackageReference Include="Microsoft.Azure.Cosmos" Version="3.35.4" />
Describe the bug
A JsonSerializationException is occurring due to the detection of a self-referencing loop during JSON serialization. This issue is specifically related to the DeclaringType property of type Microsoft.OData.Edm.EdmComplexType within the object structure.
When the Cosmos SDK attempts to parse and serialize the EdmModel, it gets stuck in a referenced loop caused by the DeclaringType field.
Various attempts have been made to resolve the issue, including adding AddODataNewtonsoftJson as mentioned in issues #749 and #774. However, these attempts have not changed the defined serialization behavior in the SDK.
Additionally, trying to ignore the error leads to excessive memory consumption as the SDK continues serializing the loop. Adjusting the MaxDepth property on JsonSerializerSettings to different values, such as 5, does not seem to have any effect.
Reproduce steps
Program.cs
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers().AddOData(options =>
options.EnableQueryFeatures(50).AddRouteComponents("odata", new MdsPeopleDataModel().GetEntityDataModel()))
.AddODataNewtonsoftJson();
var app = builder.Build();
app.UseHttpsRedirection();
app.MapControllers();
app.Run();
public class MdsPeopleDataModel
{
public IEdmModel GetEntityDataModel()
{
ODataConventionModelBuilder builder = new();
builder.ComplexType<User>();
builder.EntitySet<Employee>("Employees");
return builder.GetEdmModel();
}
}
EmployeesController.cs
[Route("[controller]")]
public class EmployeesController : ODataController
{
[EnableQuery]
public IEnumerable<Employee> Get()
{
string endpoint = "https://localhost:8081";
string authKey = "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
CosmosClient client = new CosmosClient(endpoint, authKey);
Container? container = client.GetContainer("DummyDb", "DummyContainer");
IOrderedQueryable<Employee>? query = container.GetItemLinqQueryable<Employee>(true);
return query;
}
}
Data Model
public class Employee
{
public string id { get; set; }
public User User { get; set; }
public string _rid { get; set; }
public string _self { get; set; }
public string _etag { get; set; }
public string _attachments { get; set; }
public int _ts { get; set; }
}
public class User
{
public string id { get; set; }
public string Name { get; set; }
public ICollection<Email> Emails { get; set; }
}
public class Email
{
public string EmailAdress { get; set; }
public string Type { get; set; }
}
EDM (CSDL) Model
<?xml version="1.0" encoding="utf-8"?>
<edmx:Edmx Version="4.0" xmlns:edmx="http://docs.oasis-open.org/odata/ns/edmx">
<edmx:DataServices>
<Schema Namespace="Default" xmlns="http://docs.oasis-open.org/odata/ns/edm">
<ComplexType Name="User">
<Property Name="id" Type="Edm.String" />
<Property Name="Name" Type="Edm.String" />
<Property Name="Emails" Type="Collection(Default.Email)" />
</ComplexType>
<EntityType Name="Employee">
<Key>
<PropertyRef Name="id" />
</Key>
<Property Name="id" Type="Edm.String" Nullable="false" />
<Property Name="User" Type="Default.User" />
<Property Name="_rid" Type="Edm.String" />
<Property Name="_self" Type="Edm.String" />
<Property Name="_etag" Type="Edm.String" />
<Property Name="_attachments" Type="Edm.String" />
<Property Name="_ts" Type="Edm.Int32" Nullable="false" />
</EntityType>
<ComplexType Name="Email">
<Property Name="EmailAdress" Type="Edm.String" />
<Property Name="Type" Type="Edm.String" />
</ComplexType>
<EntityContainer Name="Container">
<EntitySet Name="Employees" EntityType="Default.Employee" />
</EntityContainer>
</Schema>
</edmx:DataServices>
</edmx:Edmx>
Entity
{
"id": "1",
"User": {
"id": "1",
"Name": "John",
"Emails": [
{
"EmailAdress": "[email protected]",
"Type": "A"
}
]
},
"_rid": "jqYgAN5HGIMBAAAAAAAAAA==",
"_self": "dbs/jqYgAA==/colls/jqYgAN5HGIM=/docs/jqYgAN5HGIMBAAAAAAAAAA==/",
"_etag": "\"00000000-0000-0000-eac5-74b845c101d9\"",
"_attachments": "attachments/",
"_ts": 1695106177
}
Request/Response
https://localhost:5001/odata/employees?select=user
Expected behavior
The desired outcome is to permit the projection of solely the chosen User or its internal properties, all while avoiding the JSON serialization problem, or alternatively, enabling the option to exclude the internal properties from the EdmModel.
@devbrsa we are not seeing the self-referencing complex type in the model that you provided. Did you remove it for the sake of compiling or something like that?
Assigning to @ElizabethOkerio, please see if the repro works regardless with the provided model.
It seems it goes to non-odata routing. Since [Route("[controller]")] is not supported in OData routing, it goes to non-odata routing.
@ElizabethOkerio
@xuzhg got same behavior removing the route attribute.
@corranrogue9 below you can find the stacktrace for the self-referencing.
Newtonsoft.Json.JsonSerializationException: Self referencing loop detected for property 'DeclaringType' with type 'Microsoft.OData.Edm.EdmComplexType'. Path 'TypedProperty.SchemaElements[0].DeclaredProperties[0]'.
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CheckForCircularReference(JsonWriter writer, Object value, JsonProperty property, JsonContract contract, JsonContainerContract containerContract, JsonProp
erty containerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberC
ontract, Object& memberValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty
containerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty con
tainerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProper
ty containerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty con
tainerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty
containerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty con
tainerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProper
ty containerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty con
tainerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty
containerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty con
tainerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty
containerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty con
tainerProperty)
at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
at Newtonsoft.Json.JsonSerializer.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Type type, JsonSerializerSettings settings)
at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, JsonSerializerSettings settings)
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitConstant(ConstantExpression inputExpression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\Expres
sionToSQL.cs:line 802
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression inputExpression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\
Linq\ExpressionToSQL.cs:line 263
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3
\Microsoft.Azure.Cosmos\src\Linq\ExpressionToSQL.cs:line 1078
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.
Azure.Cosmos\src\Linq\ExpressionToSQL.cs:line 1538
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\ExpressionT
oSQL.cs:line 1452
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMemberAccess(MemberExpression inputExpression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\Expr
essionToSQL.cs:line 831
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression inputExpression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\
Linq\ExpressionToSQL.cs:line 267
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3
\Microsoft.Azure.Cosmos\src\Linq\ExpressionToSQL.cs:line 1078
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.
Azure.Cosmos\src\Linq\ExpressionToSQL.cs:line 1538
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\ExpressionT
oSQL.cs:line 1452
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMemberAssignment(MemberAssignment inputExpression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\
ExpressionToSQL.cs:line 886
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitBinding(MemberBinding binding, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\ExpressionToSQL.cs:l
ine 341
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitBindingList(ReadOnlyCollection`1 inputExpressionList, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Li
nq\ExpressionToSQL.cs:line 908
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMemberInit(MemberInitExpression inputExpression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\Ex
pressionToSQL.cs:line 966
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression inputExpression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\
Linq\ExpressionToSQL.cs:line 276
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3
\Microsoft.Azure.Cosmos\src\Linq\ExpressionToSQL.cs:line 1078
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.
Azure.Cosmos\src\Linq\ExpressionToSQL.cs:line 1538
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(LambdaExpression lambda, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\Expressio
nToSQL.cs:line 1434
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitSelect(ReadOnlyCollection`1 arguments, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\ExpressionTo
SQL.cs:line 1663
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMethodCall(MethodCallExpression inputExpression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\Ex
pressionToSQL.cs:line 1213
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.Translate(Expression inputExpression, TranslationContext context) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\ExpressionToSQL.cs
:line 135
at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.TranslateQuery(Expression inputExpression, IDictionary`2 parameters, CosmosLinqSerializerOptions linqSerializerOptions) in D:\repos\PoC\azure-cosmos-dotnet
-v3\Microsoft.Azure.Cosmos\src\Linq\ExpressionToSQL.cs:line 105
at Microsoft.Azure.Cosmos.Linq.SqlTranslator.TranslateQuery(Expression inputExpression, CosmosLinqSerializerOptions linqSerializerOptions, IDictionary`2 parameters) in D:\repos\PoC\azure-cosmos-dotnet-v
3\Microsoft.Azure.Cosmos\src\Linq\SQLTranslator.cs:line 51
at Microsoft.Azure.Cosmos.Linq.DocumentQueryEvaluator.HandleMethodCallExpression(MethodCallExpression expression, IDictionary`2 parameters, CosmosLinqSerializerOptions linqSerializerOptions) in D:\repos
\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\DocumentQueryEvaluator.cs:line 96
at Microsoft.Azure.Cosmos.Linq.DocumentQueryEvaluator.Evaluate(Expression expression, CosmosLinqSerializerOptions linqSerializerOptions, IDictionary`2 parameters) in D:\repos\PoC\azure-cosmos-dotnet-v3\
Microsoft.Azure.Cosmos\src\Linq\DocumentQueryEvaluator.cs:line 31
at Microsoft.Azure.Cosmos.Linq.CosmosLinqQuery`1.CreateFeedIterator(Boolean isContinuationExpected) in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\CosmosLinqQuery.cs:line 219
at Microsoft.Azure.Cosmos.Linq.CosmosLinqQuery`1.GetEnumerator()+MoveNext() in D:\repos\PoC\azure-cosmos-dotnet-v3\Microsoft.Azure.Cosmos\src\Linq\CosmosLinqQuery.cs:line 111
at Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.WriteResourceSetAsync(IEnumerable enumerable, IEdmTypeReference resourceSetType, ODataWriter writer, ODataSerializerContext writeContex
t)
at Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.WriteObjectInlineAsync(Object graph, IEdmTypeReference expectedType, ODataWriter writer, ODataSerializerContext writeContext)
at Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.WriteObjectAsync(Object graph, Type type, ODataMessageWriter messageWriter, ODataSerializerContext writeContext)
at Microsoft.AspNetCore.OData.Formatter.ODataOutputFormatterHelper.WriteToStreamAsync(Type type, Object value, IEdmModel model, ODataVersion version, Uri baseAddress, MediaTypeHeaderValue contentType, HttpRequest req
uest, IHeaderDictionary requestHeaders, IODataSerializerProvider serializerProvider)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isC
ompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)
@devbrsa I've created the following controller to try and repro your issue:
public class EmployeesController : ODataController
{
List<Employee> employees = new List<Employee>()
{
new Employee()
{
id = "1",
User = new User
{
id = "1",
Name = "John",
Emails = new List<Email>()
{
new Email
{
EmailAdress = "[email protected]",
Type = "A"
}
}
},
_rid = "jqYgAN5HGIMBAAAAAAAAAA==",
_self = "dbs/jqYgAA==/colls/jqYgAN5HGIM=/docs/jqYgAN5HGIMBAAAAAAAAAA==/",
_etag = "\"00000000-0000-0000-eac5-74b845c101d9\"",
_attachments = "attachments/",
_ts = 1695106177
}
};
[EnableQuery]
public IEnumerable<Employee> Get()
{
return employees;
}
}
Sending this request: https://localhost:7142/odata/Employees
I get the following response:
{
"@odata.context": "https://localhost:7142/odata/$metadata#Employees",
"value": [
{
"id": "1",
"_rid": "jqYgAN5HGIMBAAAAAAAAAA==",
"_self": "dbs/jqYgAA==/colls/jqYgAN5HGIM=/docs/jqYgAN5HGIMBAAAAAAAAAA==/",
"_etag": "\"00000000-0000-0000-eac5-74b845c101d9\"",
"_attachments": "attachments/",
"_ts": 1695106177,
"User": {
"id": "1",
"Name": "John",
"Emails": [
{
"EmailAdress": "[email protected]",
"Type": "A"
}
]
}
}
]
}
This seems to work but I'm not using Cosmos like you are. This could be an issue with Cosmos. We are looking into this. It seems like the Cosmos query provider cannot properly translate the expression that OData generates into a query when you use $select. I'm curious to know whether this works without the $select query option.
Hi @ElizabethOkerio,
Thanks for your quick reply.
The request works fine without the $select operator. I get an error when I use $select only.
@ElizabethOkerio, do you think I should open an issue in the Cosmos SDK repo?
I think you should. Which Cosmos API are you using? NoSql? PostgreSQL?
Ok. We'll also look into this but I think you should also raise the issue with the Cosmos team.
It seems it goes to non-odata routing. Since [Route("[controller]")] is not supported in OData routing, it goes to non-odata routing.
For tracking:
- https://github.com/OData/AspNetCoreOData/issues/430
@devbrsa
@xuzhg got same behavior removing the route attribute.
You are not supposed to just remove it, but replace it with the actual controller prefix manually. Not sure if you already resolved this but just mentioning in case anyone else runs into it.
