dotnet
Query: Improve query test matrix
Historically we have a large number of regressions in our LINQ implementation on every release. A significant number of regressions are introduced when we improve the stack to recognize more expressions that they can be push down to the server, but some of them are not translated correctly.
There is an ongoing brainstorming on how to improve this, and the resulting plan will be part of the focus in 2.2.
LINQ features
Query 'root'
- DbSet/Keyless DbSet
- Same/different context
- Navigation
- FromSQL
- Composable/not composable
- Mapped function
Where
- Special consideration: null semantics
- Boolean columns (as opposed to search conditions)
- Equality
- Scalar
- Structural
- Composite
- Entity (navigation, QSRE, PK to FK)
- Expressions (method calls, etc.)
OrderBy
- Complex (not on a property)
- Multiple (ThenBy)
- Duplicate (same property twice)
- Repeated (first one ignored)
Select
- Single element X
- Collection of X
- Where X in
- Anonymous
- DTO
- Tuple/ValueTuple
- Entity/Keyless type
- Scalar
- And possibly recursively...
SelectMany
- Anonymous
- DTO
- Tuple/ValueTuple
- Entity/Keyless type
- Scalar
- With DefaultIfEmpty
- With explicit result selector
- And possibly recursively...
Join/GroupJoin
- Special consideration: combine with projection
- Property
- Entity
- Navigation (left/right key selector)
- LOJ pattern (GroupJoin, SelectMany, DefaultIfEmpty)
GroupBy
- Naked
- With aggregate (scalar/expression/sub-query/VB)
- With/without result selector
- Different group keys
- Constant
- Scalar
- Entity
- Composite
- With/without element selector
Result operators
- Count/LongCount
- Min/Max
- Sum/Average
- Skip/Take
- Distinct
- First/FirstOrDefault/Single/SingleOrDefault
- Last*
- OfType/Cast (inheritance)
- All/Any
- DefaultIfEmpty
- Set (Union, etc.)
- SkipWhile/TakeWhile
- Contains
- Reverse
Expressions
- Contains
- DateAdd
- DateDiff
- StartsWith/EndsWith
- Like
- Math.*
- ToArray/ToList/etc.
- AsEnumerable/AsQueryable
- String manipulation
- Concatenation
- IndexOf
- IsNullOrEmpty/WhiteSpace
- Length
- Replace
- Substring
- ToLower/ToUpper
- Trim/TrimEnd/TrimStart
- Contains/StartsWith/EndsWith
- String comparison modifiers
Other operations
- Coalesce
- Ternary conditional
- Type conversion
- Implicit
- Explicit cast
- as
- is
- VB converts
‘let’ keyword
- Funcletization
- Different complexity (literal/expression/recursive/skip/take)
- Functions always sent to the server Guid.NewID
- Closure variables
- Scalar
- Entity
- Collection (in case of Contains)
Navigation expressions
- Reference
- Collection
- Skip collection
- Multi level
- PK -> FK optimization (order.Customer.Id -> order.CustomerId)
- Special consideration:
- Owned
- Inside Join key selector
- Inside SelectMany
- Optional/required
Include
- Reference/Collection/Skip collection
- Split/Single
- Nested
- On a derived type
- String
- Filtered
Sync/Async
AsTracking/AsNoTracking
Scenarios
Correlated Collection
- Variation of all basic query concepts: Where, OrderBy, Select
- Nested,
- Collection returning only one scalar (not currently handled well)
Complex binding
- Reference element from outside of a subquery
Null semantics
- Simple
- Negated
- Complex (coalesce, conditional)
Streaming/buffering
SelectExpression
- Type inference
- RowNumberPaging
Client evalution
EF.Property
MARS
Model features
Entity Types
- Simple
- Derived
- Owned
- Weak
- Derived
- Shared type
- Keyless
Functions
- Built-in
- Provider specific
Properties
- Optional/Required
- Type Conversion
- Simple (int -> string)
- Structural (byte[] -> long, JSON -> string)
- Shadow/non-shadow
Keys
- string/GUID/int
- byte[]
- Composite
- Alternate
- int[] and other types with value converters
- Composite over discriminator
Foreign keys
- Optional/Required
- Unique/Nonunique
- Self-referencing
- 0/1/2 navigations
Mapping
- TPH/Complete TPH/TPT/TPC/Entity splitting
- Table sharing
- Shared columns
- View/Defining Query/TVF
Global filters
For the entity class features see https://github.com/aspnet/EntityFrameworkCore/issues/12014
Just making sure these are covered explicitly:
- equality in general, e.g. scalar vs. structural vs. composite vs. entity (this is somewhat covered in groupby and join keys , but it applies to other parts of the query)
- model-defined (global) filters
- defining query
- query types
- usage of DTOs, anonymous, Tuples inside the query (although I assume these are implicitly covered as just a combination of projection plus some other operator applied afterwards)
Do we need to call out specific LINQ query operators in this list or should we assume that all query operators in Queryable and Enumerable are fair game?
Also, I wonder if type mapping and value conversions is something we should add to the mix here (or in https://github.com/aspnet/EntityFrameworkCore/issues/12014) or keep as separate. I believe there are types that are particularly interesting, like binary.
The Visual Basic Aggregate Clause can produce some interesting expressions that may be worth adding coverage for.
As a user that sis bitten by 2.1.1 basically blowing every non trivial query we have in ef6 - can you please put a focus on a struture/frameowkr that allows end users to submit test cases? And then actually put a focus on fixing those, not just ignoring them for a grand larger plan?
We moved from ef6 to 2.1.1 a month ago, and I am stuck with half my queries literally blowing in my face with all kinds of issues, from bad sql generated to parsing exceptions. I have test cases for most thigns that I could put up as repro, but that is way too much ramp up and no documentation how to do that easily.
As long as you focus on some individual isolated expressions there is always the risk of overlooking complex interactions - and those are coming from users.
I applause the attempt to get LINQ into a less error prone scenario, but this also must include an easy ramp for end user cases that then get fixed - and the fix published, preferably not in the "next marjor release which, btw., you can use possibly in 4 months because hey, we don't really want you to get your work into your production".
If this sounds frustrated - it is. Stuck on 2.1.1 with basics not working and no release date. Heck, today i found one (https://github.com/aspnet/EntityFrameworkCore/issues/12951) but I lack the documentaiton and guideliens how to put up a test case for it. Stuff like that can make end users provide really complex text cases. Oh, yeah - you likely try aout a lot of permutations. My cases possibly involve 8-10 navigations and each of it hsa up to 2 global filters that possibly add another 4-5 table joins. This is where you sometimes find really interesting things.
Moving to the 3.0 milestone for tracking; it's not really punted, just ongoing work.
Initial work and infrastructure added in cb270316d63730ae669ca8b6b8fb16b2a2f6abc2
Verification will be added in the future - mutators can be in 2 modes: generating query and generating baseline. When ran with the same seed the resulting query should be the same, unless the baseline query was different in the first place (need to compensate for Maybe<T> but appart from that, most baselines are identical.
We also need to create a better comparer for untyped results, The one we currently have simply calls to underlying Equals method which won't work for collections and other more complicated structures.
Removing from milestone to discuss the relevance after we've disabled client eval.
