Strange memory behaviour with large number of columns

[Steve887]

I have run into a strange issue with our app that results in strange memory usage: spiky memory loads and increasing memory usage over time, when there is a large number of properties in the model being selected. What's particularly strange, if I comment out just one property, the memory issues no longer occur and memory usage is extremely consistent.

A requirement of our system is to change connection strings at runtime, so instead of using Services.AddDbContext, I am registering the data context with Autofac and passing in a connection string at runtime, then using an overridden OnConfiguring to setup the SqlServer provider. If I change this to use Services.AddDbContext then memory does return the normal, but I'm confused why this would only seem to have an effect at a large number of properties.

The rest of my setup is very normal, with a new entities being selected with Includes.

I have attached an app that reproduces the issue, the steps are as follows:

1. Open the memory test solution
2. Start the MemoryTest.Api project. This will create a database on a (localdb)\MSSQLLocalDB database, so change this for SqlExpress or other server etc in Program.cs.
3. Start memory profiling with your favourite program (I used dotMemory)
4. Run the StressTestApi.ps1 file. This will execute an API call against the endpoint constantly to simulate a load.

Running the application as is, results in a memory graph as follows:

Open Item.cs and comment out the ItemImage property. Then open ImageMap.cs and comment out the ItemImage property mapping. Start the application again, attach the memory profiler and rerun the powershell script. This results in the following memory graph:

The big differences seem to be in the gen 1 and 2 heaps, although taking memory snapshots doesn't really reveal anything obvious. The total memory also grows over time when the column is there.

While it's easy enough to say, just decrease the model size, I am working with a large, legacy, model and cannot make big changes like that. I would also like the know the underlying reason why the memory behaviour changes so drastically just by changing one column. I would expect if the setup is wrong for it to happen all the time.

This also occurs in .Net 7 and EFCore 7 versions, on windows and linux.

Please let me know if there's any more information to supply.

MemoryTest.zip

Include provider and version information

EF Core version: 8.0.8 Database provider: Microsoft.EntityFrameworkCore.SqlServer Target framework: .Net 8 Operating system: Windows