Injecting a DBContext with AddDbContext and UseNpgsql causes it to remain in memory after the scope has been disposed of

[paulnsk]

I am new to Postgres so if this behavior is somehow by design, I apologize.

Here is my DBContext with nothing in it

    public class MyContextForInjection(DbContextOptions<MyContextForInjection> options) : DbContext(options)
    {

    }

Here is me registering it with for the dependency injection as a scoped service:

  var builder = Host.CreateApplicationBuilder();
  builder.Services.AddDbContext<MyContextForInjection>
  (
      options =>
      {    
          options.UseNpgsql("foo");
      }
  );

Note the bogus connection string, so no actual connection to a DB is even being made.

Now let's try using it:

            var app = builder.Build();
            var serviceProvider = app.Services.GetRequiredService<IServiceProvider>();
            using (var skope = serviceProvider.CreateScope())
            {
                using (var myContext = skope.ServiceProvider.GetRequiredService<MyContextForInjection>())
                {
                    Console.WriteLine("DB context created!");
                }
            }
            Console.WriteLine("Press Enter to exit...");
//---------->   at this point an instance of MyContextForInjection is still in memory
            Console.ReadLine();

While the app is waiting for ReadLine() go to Visual Studio Diagnostic tools, take memory snapshot and inspect it.

This does not happen with MS SQL (UseSqlServer). This does not happen when the DBContext is newed up manually (in which case UseNpgSql is called from its OnConfiguring() method).

Here is a sample project. UseNpgSqlBug.zip

In real life, when an actual connection to a DB is made followed by some data retrieval or insertion, all the tracked entitites remain in memory, in my case it was hundreds of megabytes. Curiously though, if I put the whole using block in a loop, I don't see multiple instances of the DBContext, just one. So it behaves kind of like a singleton (or maybe the old one does get destroyed when a new one is created).

[paulnsk]

After a bit more digging, found this todo item. Applying it seems to have fixed the issue. It was NpgsqlSingletonOptions.ApplicationServiceProvider that was holding a reference to my db context instance preventing it from being garbage collected. Wondering if there is any reason why this todo was never done, perhaps simply missed. That PR has long since been merged.

[roji]

@paulnsk I've looked at your sample code, and I'm not quite following... The fact that a DbContext instance shows up in memory after it was disposed is expected: .NET uses a garbage collector which kicks in at arbitrary times, and objects can stay in memory for a long time after they're disposed and unreachable before they get collected. The question is whether the objects can get collected, i.e. whether they're still rooted (referenced) in some way that prevents the garbage collector from collecting them.

I put together a quick sample based on your code that repeatedly creates a new scope and context in a loop (see code below), and ran that with a memory profiler; as expected, memory usage is very stable and no memory leak is apparent.

Code sample

var builder = Host.CreateApplicationBuilder();
builder.Services.AddDbContext<MyContextForInjection>
(
    options =>
    {
        //bug!!
        options.UseNpgsql("foo");

        //no bug
        //options.UseSqlServer("foo");
    }
);


using var app = builder.Build();

for (var i = 0; i < 1000000; i++)
{
    var serviceProvider = app.Services.GetRequiredService<IServiceProvider>();
    using (var skope = serviceProvider.CreateScope())
    {
        using (var myContext = skope.ServiceProvider.GetRequiredService<MyContextForInjection>())
        {
        }
    }
}

public class MyContextSimple:  DbContext
{
    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        base.OnConfiguring(optionsBuilder);
        optionsBuilder.UseNpgsql("foo");
    }
}

public class MyContextForInjection(DbContextOptions<MyContextForInjection> options) : DbContext(options);

In real life, when an actual connection to a DB is made followed by some data retrieval or insertion, all the tracked entitites remain in memory, in my case it was hundreds of megabytes.

Can you try to reproduce this in a minimal, runnable code sample?

[paulnsk]

they're still rooted (referenced) in some way that prevents the garbage collector from collecting them

Yes, it is referenced and hence not collected by GC. Otherwise it would only be shown by VS memory profiler when "Show dead objects" is checked. In fact, it is referenced specifically by NpgsqlSingletonOptions.ApplicationServiceProvider and that's why removing this line ApplicationServiceProvider = coreOptions.ApplicationServiceProvider; fixed the issue for me. (The line is marked for removal anyway, by the todo item on top of it) Putting scope creation in a loop reveals nothing because only one rooted instance of a dbcontext remains in memory. So, even though the total amount of memory would appear stable, one copy of dbcontext which is never GCed can become a big deal if you are using it to bulk insert 100K records, which is what I was doing and this is how I discovered the issie.

[roji]

@paulnsk OK, thanks for the added info, I'll look deeper into this.

[sbjessee]

I think I'm running into this issue as well, the npgsql is creating a ton of SQL strings and parameters for my use case, but the garbage collector is very hesitant to collect it and behaves like a memory leak. I'm running my solution in a container with limited resources, so it eventually thrashes. If I use JetBrains DotMemory to profile it, it can successfully invoke a garbage collection, but I'm not able to replicate that with GC code or through the Visual Studio profiler.

[roji]

@sbjessee what you describe is not a leak, and nothing that either Npgsql or EF can control; either an object is still rooted (referenced by something), in which case the GC cannot reclaim the memory, or it isn't, and then it's up to the GC to decide when it gets reclaimed (EF/Npgsql have no control over this). If you think the GC is misbehaving in some way, you'll have to report that on the .NET runtime repo - but with more details than the above (e.g. a repro).

[roji]

In version 9.0, NpgsqlSingletonOptions no longer references the application service provider (i.e. the "todo" line above was removed) - this was done as a more general cleanup of how DbDataSource and Npgsql-level configuration is managed in the EF provider (see the 9.0 release notes).

So I'll go ahead and close this issue for now - thanks for flagging!