EFCore.BulkExtensions icon indicating copy to clipboard operation
EFCore.BulkExtensions copied to clipboard

BulkInsert not working

Open Woudjee opened this issue 1 year ago • 9 comments

Why does this work?

List<BrepEntity> brepjes = new List<BrepEntity>();

for (int i = 0; i < 100; i++) { brepjes.Add(new BrepEntity() { Polygons = new List<PolygonEntity>() { new PolygonEntity() { Vertices = new List<VectorEntity>() { new VectorEntity() { X = 123, Y = 2348.324f, Z = 234293784.1f }, } }, new PolygonEntity() { Vertices = new List<VectorEntity>() { new VectorEntity() { X = 334, Y = 244648.324f, Z = 909793784.1f }, } } } }); }

Stopwatch stopwatch = Stopwatch.StartNew(); _context.BulkInsert(brepjes, options => options.IncludeGraph = true); _context.BulkSaveChanges();

stopwatch.Stop(); return stopwatch.ElapsedMilliseconds;

But this does not?

List<GridInfoEntity> test = new List<GridInfoEntity>() { new GridInfoEntity() { GridInfoX = new List<GridInfoXEntity>() { new GridInfoXEntity("Label", 123), }, GridInfoY = new List<GridInfoYEntity>() { new GridInfoYEntity("Label", 456), }, GridInfoZ = new List<GridInfoZEntity>() { new GridInfoZEntity("Label", 789), }, } };

Stopwatch stopwatch = Stopwatch.StartNew(); _context.BulkInsert(test, options => options.IncludeGraph = true); _context.BulkSaveChanges();

stopwatch.Stop(); return stopwatch.ElapsedMilliseconds;

I am getting this error, which is incorrect:

Missing Column : GridInfoId On entity : GridInfoXEntity On Table : [Domain].[GridInfo]

Woudjee avatar Dec 11 '23 12:12 Woudjee

If I use AddRange for the not working example, it works. I am therefore sure that both the tables and columns exist.

Woudjee avatar Dec 11 '23 12:12 Woudjee

After 13 hours of testing and 14 minutes after making this post, I figured out the problem. BulkInsert cannot handle schemas!

[Table("GridInfo", Schema = "Domain")]

When I removed this line of code, it worked. Perhaps you want to check that out.

Woudjee avatar Dec 11 '23 12:12 Woudjee

Could you add entire code of class GridInfoEntity and GridInfoZEntity or write a Test for the issue.

borisdj avatar Dec 11 '23 14:12 borisdj

Yes, of couse. See below (screenshot for readability + copy paste):

GridInfoEntity image

[Table("GridInfo", Schema = "Domain")] public class GridInfoEntity : Entity { public virtual ICollection<GridInfoXEntity> GridInfoX { get; set; } public virtual ICollection<GridInfoYEntity> GridInfoY { get; set; } public virtual ICollection<GridInfoZEntity> GridInfoZ { get; set; } }

GridInfoXEntity image

[Table("GridInfo.X", Schema = "Domain")] public class GridInfoXEntity : Entity { public GridInfoXEntity(string label, float value) { ValueX = value; Label = label; }

public GridInfoXEntity()
{

}

public float ValueX { get; set; }
public string Label { get; set; }

public virtual GridInfoEntity GridInfo { get; set; }

}

Exactly the same applies to the Y and Z variant (I double checked in my code).

And for the Brep-chain:

BrepEntity image

public class BrepEntity : Entity { public virtual ICollection<PolygonEntity> Polygons { get; set; } }

PolygonEntity image

public class PolygonEntity : Entity { public int BrepId { get; set; } public virtual BrepEntity Brep { get; set; } public virtual ICollection<VectorEntity> Vertices { get; set; } }

VectorEntity image

public class VectorEntity : Entity { public float X { get; set; } public float Y { get; set; } public float Z { get; set; }

public int PolygonId { get; set; }
public virtual PolygonEntity Polygon { get; set; }

}

Woudjee avatar Dec 11 '23 14:12 Woudjee

The only difference appears to be the schema. Once I removed it, everything worked as expected. And I remember from last week that I got exception that actually used the schema name. It said that the schema name did not exist (while it definitely did). I guess somewhere in your code, you are not handling schemas. Is that possible?

For now, I disabled schemas in my code base (hopefully temporarily, because my database is not nicely structured anymore when viewing it in SMSS).

Woudjee avatar Dec 11 '23 14:12 Woudjee

Schemas are actually supported with main Bulk ops. But IncludeGraph often had exceptions. I will lookup into it more. In the meantime you could try to call 2 times BulkInsert, instead of using IncludeGraph.

borisdj avatar Dec 11 '23 15:12 borisdj

How do you mean that exactly? I interpreted IncludeGraph as: include all children so that all gets added. In this particular example, you mean storing the X-values, Y-values, Z-values and grid infos themselves as separate calls?

I guess that would work here, but in my actual project this would get too messy (I have referred a different post 38 minutes ago). I can live with not having schemas for the time being.

Woudjee avatar Dec 11 '23 15:12 Woudjee

Yes, that's right.

borisdj avatar Dec 11 '23 15:12 borisdj

Thank you Boris for you quick replies. I will keep an eye on this post and would be more than happy to contribute to solving this problem wherever I can. Just let me know.

Woudjee avatar Dec 11 '23 15:12 Woudjee