zzzprojects
IncludeGraph = true needs better docs / error messages
Description
I am trying to use IncludeGraph = true, but each time I run my tests, I fail with a different error.
- First time, it told me to use a context factory. Ok, done.
- Second time, it told me I had to use IncludeGraphOptionOperationBuilder. Ok, went to the include-graph web page provided in the exception link, and found a sparse example. Ok, implemented a basic example.
- Third time, it told me that when I use IncludeGraphOperationBuilder, I need to specify IgnoreOnMergeUpdateExpression and LambdaIgnoreOnMergeUpdateExpression. Exception below
After some experimentation, it appears the error message is necessary but not sufficient.
- Typo. The issue is not that these options must be specified in 'IncludeGraphBuilder' but 'IncludeGraphOperationBuilder'.
- Containment error. It's not that specifying these options in the IncludeGraphOperationBuilder fix the problem. Rather, it's that you can't specify these settings outside the IncludeGraphOperationBuilder. In Issue #296 , you reference that this is confusing, but did not improve the documentation.
- Incomplete documentation. The error message points to https://entityframework-extensions.net/include-graph and mentions you have to use IgnoreOnMergeUpdateExpression and LambdaIgnoreOnMergeUpdateExpresion, but the example given on the website doesnt use these. Initially, I thought this was simply incomplete documentation. I now realize these are only used if you have to use them. In addition to clarifying the exception message, the website documentation should be updated with a note of an example of an invalid configuration and a valid configuration.
Exception
System.Exception
When using `IncludeGraph`, some options must be set in the `IncludeGraphBuilder` (See: https://entityframework-extensions.net/include-graph). The following options must be specified in `IncludeGraphBuilder`: IgnoreOnMergeUpdateExpression, LambdaIgnoreOnMergeUpdateExpression
at Z.BulkOperations.BulkOperation`1.(BulkOperation )
at DbContextExtensions.`1.(BulkOperation )
at DbContextExtensions.(DbContext this, Boolean , Action`1 , Boolean )
at DbContextExtensions.BulkMerge[T](DbContext this, IEnumerable`1 entities, Action`1 bulkOperationFactory)
Fiddle or Project (Optional)
Context.UnderlyingContext.BulkMerge(entities, options =>
{
options.MergeKeepIdentity = true;
options.SqlBulkCopyOptions = (int)SqlBulkCopyOptions.TableLock;
options.AutoMapOutputDirection = false;
options.ContextFactory = context =>
{
var contextLocator = _lifetimeScope.Resolve<IDbContextLocator>();
return contextLocator.Locate<RawFile>().UnderlyingContext;
};
options.IncludeGraph = true; // this is different from what we did in Edgar.
options.IncludeGraphOperationBuilder = operation =>
{
// Before modifying anything inside this lambda expression, please see: https://github.com/zzzprojects/EntityFramework-Extensions/issues/296#issuecomment-541765497
// Quote from JonathanMagnan: "The IncludeGraph is not very obvious. Everything that depend on an entity must be specified in the IncludeGraphOperationBuilder."
if (operation is BulkOperation<RawFile> rawFillBulkOperation)
{
Expression<Func<ElectronicTradeFeedRawFill, object>> func = e => new { e.CreatedDateTime };
options.LambdaIgnoreOnMergeUpdateExpression = func;
}
};
if (typeof(IHasTimeStamp).IsAssignableFrom(typeof(RawFile)))
{
Expression<Func<RawFile, object>> func = e => new { e.CreatedDateTime };
options.LambdaIgnoreOnMergeUpdateExpression = func;
}
}
Otherwise, make sure to include as much information as possible to help our team to reproduce the issue.
Note: More information you provide, faster we can implement a solution.
Further technical details
- EF version: 6.4.4
- EF Extensions version: 1.12.23
- Database Provider: SQL Server
In addition, it seems as though it is invalid/illogical to specify "InsertKeepIdentity" on a BulkOperation<T> in an IncludeGraphOperationBuilder for a BulkMerge. The Insert is referring to the Zzz Projects instead, as opposed to the T-SQL Insert. Therefore, it seems correct to use MergeKeepIdentity. Please confirm.
Also, is there an easy way I can spy on what stream of data is being sent to SQL Server? SQL Profiler only shows the bulk insert as a bulk insert with no data. I'm worried with all these graph options that I'll configure something in a subtle way, and, worse, somebody later on will discover a bug and won't be able to figure it out from the documentation.
Hello @jzabroski ,
Sorry for the long wait, I didn't see your second message.
We improved the error message in the version we released yesterday (v4.0.82)
- Fixed the typo
- Fixed the
LambdaIgnoreOnMergeUpdateExpresionthat should not have appeared - Improved the message
We are currently working to rewrite the page. I believe it will be released on next Monday.
You are right, MergeKeepIdentity should be used when using BulkMerge.
Unfortunately no unless only a few entities is used. When using over 10 entities, a SqlBulkCopy is done under the hood in the destination table or a temporary table, and unfortunately, there is no way (that I'm aware) to see this value.
It could be possible to "monitor" them via the Auditing features but I'm not sure that's what you really want.
One thing I could not figure out how to do was to ignore a specific entity relationship. For example,
Imagine a graph:
public class ConvertedFill
{
public int Id {get; set;}
public int File ConversionLog {get;set;}
}
public class RawFill
{
public int Id {get; set;} public File File {get; set;}
public ConvertedFill ConvertedFill {get; set;}
}
It is not clear to me how to use BulkOperation<File>.IsReadOnly if I want ConvertedFill.ConversionLog to be merged, but not RawFill.File to be merged.
I circumvented this requirement but I found the code a bit hard to read (not my proudest code despite the speed gains).