zzzprojects
IncludeFilter issue on .NET Core 3.1
Description
Hello
After migrating solution from .NET Core 2.2 to 3.1, my EF queries started throwing exceptions. The problem occurs when I use IncludeFilter syntax. I attached project in which im facing the issue. You can find sample code in IssueController.
Exception
Exception message:InvalidOperationException: Processing of the LINQ expression 'source => (IEnumerable<SkillName>)source' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.
Stack trace:
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
An unhandled exception has occurred while executing the request.
System.InvalidOperationException: Processing of the LINQ expression 'source => (IEnumerable<SkillName>)source' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitatio
n in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.
at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.ProcessSelectMany(NavigationExpansionExpression source, LambdaExpression collectionSelector, LambdaExpression resultSe
lector)
at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
at Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.Expand(Expression query)
at Microsoft.EntityFrameworkCore.Query.QueryTranslationPreprocessor.Process(Expression query)
at ?.?(IQueryable ?, Action`1 ?, RelationalQueryContext& ?, Object& ?)
at Z.EntityFramework.Extensions.EFPlusExtensions.EFPlusCreateCommand(IQueryable source, Action`1 action, RelationalQueryContext& queryContext, Object& compiledQuery)
at Z.EntityFramework.Plus.BaseQueryFuture.CreateExecutorAndGetCommand(RelationalQueryContext& queryContext)
at Z.EntityFramework.Plus.QueryFutureBatch.CreateCommandCombined()
at Z.EntityFramework.Plus.QueryFutureBatch.ExecuteQueries()
at Z.EntityFramework.Plus.QueryFutureEnumerable`1.GetEnumerator()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at Z.EntityFramework.Plus.QueryIncludeFilterParentQueryable`1.CreateEnumerable()
at Z.EntityFramework.Plus.QueryIncludeFilterParentQueryable`1.GetEnumerator()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at IncludeFilterIssue.Controllers.WeatherForecastController.Issue() in C:\Users\xdelp\Desktop\Nowy folder\IncludeFilterIssue\IncludeFilterIssue\Controllers\WeatherForecastController.cs:line 41
at IncludeFilterIssue.Controllers.WeatherForecastController.Get() in C:\Users\xdelp\Desktop\Nowy folder\IncludeFilterIssue\IncludeFilterIssue\Controllers\WeatherForecastController.cs:line 31
at lambda_method(Closure , Object , Object[] )
at Microsoft.Extensions.Internal.ObjectMethodExecutor.Execute(Object target, Object[] parameters)
at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
Fiddle or Project
https://github.com/del77/IncludeFilterIssue
Further technical details
- EF version: EF Core 3.1.3
- EF Plus version: 3.0.45
- Database Provider: SQL Server 15.0.2070
Thank you for the project,
We successfully reproduced the issue. We will look if we can do something about it.
Best Regards,
Jon
Performance Libraries
context.BulkInsert(list, options => options.BatchSize = 1000);
Entity Framework Extensions • Entity Framework Classic • Bulk Operations • Dapper Plus
Runtime Evaluation
Eval.Execute("x + y", new {x = 1, y = 2}); // return 3
C# Eval Function • SQL Eval Function
Hello all,
Sorry for the long waiting,
We believe we finally find out how to fix this issue.
The v3.0.60 has been released today with a fix which remove the cast that was working under EF Core 2.x but was throwing starting from EF Core 3.x
If anyone has the chance to test it, let us know if everything is working.
This case from the project I provided has been solved by this update. Unfortunately, extended case is still throwing for me.
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
An unhandled exception has occurred while executing the request.
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.ArgumentNullException: Value cannot be null. (Parameter 'source')
at System.Linq.Queryable.SelectMany[TSource,TResult](IQueryable`1 source, Expression`1 selector)
at Z.EntityFramework.Plus.QueryIncludeFilterManager.SelectManyFutureNoCast[IEntity](IQueryable`1 query)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
at Z.EntityFramework.Plus.QueryIncludeFilterChild`2.CreateIncludeQuery(IQueryable rootQuery)
at Z.EntityFramework.Plus.QueryIncludeFilterParentQueryable`1.CreateEnumerable()
at Z.EntityFramework.Plus.QueryIncludeFilterParentQueryable`1.GetEnumerator()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at IncludeFilterIssue.Controllers.WeatherForecastController.IssueExtended() in C:\Users\xdelp\RiderProjects\IncludeFilterIssue\IncludeFilterIssue\Controllers\WeatherForecastC
ontroller.cs:line 56
at IncludeFilterIssue.Controllers.WeatherForecastController.Get() in C:\Users\xdelp\RiderProjects\IncludeFilterIssue\IncludeFilterIssue\Controllers\WeatherForecastController.
cs:line 33
at lambda_method(Closure , Object , Object[] )
at Microsoft.Extensions.Internal.ObjectMethodExecutor.Execute(Object target, Object[] parameters)
at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object control
ler, Object[] arguments)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object
state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
I hope so,
However, my developer that did the latest change, unfortunately, is not available till next Monday, so I was waiting for him to come back to follow this request with him
Any update about this issue? We are migrating our application from 2.2 and we are having the same problem with some queries.
Unfortunately @rvicoqbi ,
We didn't succeed to fix it.
We will have to fully re-write this feature and doing it will highly depend on how well work the filtered include now part of EF Core 5: https://www.learnentityframeworkcore5.com/whats-new-in-ef-core-5/filtered-included
@rvicoqbi, it turned out that _context.Courses.IncludeFilter(c => c.SkillCourses.Select(sc => sc.Skill.Names)).ToList(); from the repository I provided is the only one case which doesn't work after last fixes. Therefore I decided that until we can move to EF Core 5 which should support include filtering, I will use the following workaround:
Instead of:
_context.Courses.IncludeFilter(c => c.SkillCourses.Select(sc => sc.Skill.Names)).ToList();
I used:
_context.SkillNames.Future();
_context.Courses.IncludeFilter(c => c.SkillCourses.Select(sc => sc.Skill)).ToList();
In this case Skill.Names will be set by context.
That's exactly our case @del77 but we have bigger and more complex queries similar to that. So, let's try with your workaround and let's see if we can save this ball.
Yeah @JonathanMagnan, we are aware of EF Core 5 new functionalities and thinking in doing the migration to 5 instead of 3.1 but we aren't totally sure for now, debate is open 😄.
Thanks so much to both of you guys.