npgsql
Mapping enum issues with multiple DbContext's
When using multiple DbContexts with the same connection string or data source and trying to .MapEnum the enums present on both of them will throw an exception at startup. We ran into this issue while trying to upgrade from net8 to net9
We are using DbContextPool and PooledDbContextFactory in our application, because in same specific parts it makes more sense to use the factory.
services.AddDbContextPool<MyContext>(builder =>
{
builder.UseNpgsql(config.Db.Conn, optionsBuilder =>
{
optionsBuilder.MapEnum<RankType>();
});
});
services.AddPooledDbContextFactory<MyContext>(builder =>
{
builder.UseNpgsql(config.Db.Conn, optionsBuilder =>
{
optionsBuilder.MapEnum<RankType>();
});
});
The same is true when using a datasource. (It really doesnt make sense to me that .MapEnum on the datasource doesnt do anything btw. not sure if this is intended. I assumed just defining it on the data source was enough to begin with)
var dataSource = new NpgsqlDataSourceBuilder(config.Db.Conn).MapEnum<RankType>().Build();
services.AddDbContextPool<MyContext>(builder =>
{
builder.UseNpgsql(dataSource, optionsBuilder =>
{
optionsBuilder.MapEnum<RankType>();
});
});
services.AddPooledDbContextFactory<MyContext>(builder =>
{
builder.UseNpgsql(dataSource, optionsBuilder =>
{
optionsBuilder.MapEnum<RankType>();
});
});
The error that is being thrown
[11:19:13.489] [ERR] [Microsoft.Extensions.Hosting.Internal.Host] Hosting failed to start
System.InvalidOperationException: Sequence contains more than one matching element
at System.Linq.ThrowHelper.ThrowMoreThanOneMatchException()
at System.Linq.Enumerable.TryGetSingle[TSource](IEnumerable`1 source, Func`2 predicate, Boolean& found)
at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlTypeMappingSource.FindEnumMapping(RelationalTypeMappingInfo& mappingInfo)
at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlTypeMappingSource.FindMapping(RelationalTypeMappingInfo& mappingInfo)
at Microsoft.EntityFrameworkCore.Storage.RelationalTypeMappingSource.<>c.<FindMappingWithConversion>b__8_0(ValueTuple`4 k, RelationalTypeMappingSource self)
at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd[TArg](TKey key, Func`3 valueFactory, TArg factoryArgument)
at Microsoft.EntityFrameworkCore.Storage.RelationalTypeMappingSource.FindMappingWithConversion(RelationalTypeMappingInfo mappingInfo, Type providerClrType, ValueConverter customConverter)
at Microsoft.EntityFrameworkCore.Storage.RelationalTypeMappingSource.FindMapping(MemberInfo member, IModel model, Boolean useAttributes)
at Microsoft.EntityFrameworkCore.Metadata.Internal.MemberClassifier.IsCandidatePrimitiveProperty(MemberInfo memberInfo, IConventionModel model, Boolean useAttributes, CoreTypeMapping& typeMapping)
at Microsoft.EntityFrameworkCore.Metadata.Conventions.PropertyDiscoveryConvention.IsCandidatePrimitiveProperty(MemberInfo memberInfo, IConventionTypeBase structuralType, CoreTypeMapping& mapping)
at Microsoft.EntityFrameworkCore.Metadata.Conventions.PropertyDiscoveryConvention.DiscoverPrimitiveProperties(IConventionTypeBaseBuilder structuralTypeBuilder, IConventionContext context)
at Microsoft.EntityFrameworkCore.Metadata.Conventions.PropertyDiscoveryConvention.ProcessEntityTypeAdded(IConventionEntityTypeBuilder entityTypeBuilder, IConventionContext`1 context)
at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ImmediateConventionScope.OnEntityTypeAdded(IConventionEntityTypeBuilder entityTypeBuilder)
at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.OnEntityTypeAddedNode.Run(ConventionDispatcher dispatcher)
at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.DelayedConventionScope.Run(ConventionDispatcher dispatcher)
at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ConventionBatch.Run()
at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ConventionBatch.Dispose()
at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ImmediateConventionScope.OnModelInitialized(IConventionModelBuilder modelBuilder)
at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.ImmediateConventionScope.OnModelInitialized(IConventionModelBuilder modelBuilder)
at Microsoft.EntityFrameworkCore.Metadata.Conventions.Internal.ConventionDispatcher.OnModelInitialized(IConventionModelBuilder modelBuilder)
at Microsoft.EntityFrameworkCore.Metadata.Internal.Model..ctor(ConventionSet conventions, ModelDependencies modelDependencies, ModelConfiguration modelConfiguration)
at Microsoft.EntityFrameworkCore.ModelBuilder..ctor(ConventionSet conventions, ModelDependencies modelDependencies, ModelConfiguration modelConfiguration)
at Microsoft.EntityFrameworkCore.ModelConfigurationBuilder.CreateModelBuilder(ModelDependencies modelDependencies)
at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.CreateModel(DbContext context, IConventionSetBuilder conventionSetBuilder, ModelDependencies modelDependencies)
at Microsoft.EntityFrameworkCore.Infrastructure.ModelSource.GetModel(DbContext context, ModelCreationDependencies modelCreationDependencies, Boolean designTime)
at Microsoft.EntityFrameworkCore.Internal.DbContextServices.CreateModel(Boolean designTime)
at Microsoft.EntityFrameworkCore.Internal.DbContextServices.get_Model()
at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.<TryAddCoreServices>b__8_4(IServiceProvider p)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
at Microsoft.EntityFrameworkCore.Infrastructure.EntityFrameworkServicesBuilder.<>c.<TryAddCoreServices>b__8_9(IServiceProvider p)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitIEnumerable(IEnumerableCallSite enumerableCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
at Microsoft.EntityFrameworkCore.DbContext.Microsoft.EntityFrameworkCore.Infrastructure.IResettableService.ResetState()
at Microsoft.EntityFrameworkCore.Internal.DbContextPool`1.Return(IDbContextPoolable context)
at Microsoft.EntityFrameworkCore.Internal.DbContextLease.Release()
at Microsoft.EntityFrameworkCore.Internal.ScopedDbContextLease`1.System.IDisposable.Dispose()
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose()
at Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env, ILoggerFactory loggerFactory, ILogger`1 logger) in F:\Dev\Git\\API\API\Startup.cs:line 189
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__14_1(IHostedService service, CancellationToken token)
at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)
Can you please submit a minimal, runnable code sample? Partial snippets are very rarely enough to understand exactly what a user is doing.
Here ya go, simple as that.
using Microsoft.EntityFrameworkCore;
var builder = WebApplication.CreateBuilder(args);
const string connectionString = "Host=localhost;Port=5432;Database=root;Username=root;Password=root"; // This doesnt need to be up, just needs to be a valid conn string
builder.Services.AddDbContextPool<MyContext>(optionsBuilder =>
{
optionsBuilder.UseNpgsql(connectionString, npgsql =>
{
npgsql.MapEnum<FunnyEnum>(); // Map Enum
});
});
builder.Services.AddPooledDbContextFactory<MyContext>(optionsBuilder =>
{
optionsBuilder.UseNpgsql(connectionString, npgsql =>
{
npgsql.MapEnum<FunnyEnum>(); // Map Enum, but this seems to cause the issue
});
});
var app = builder.Build();
await using var scope = app.Services.CreateAsyncScope();
var myContext = scope.ServiceProvider.GetRequiredService<MyContext>(); // Request context to initialize
var tryToQuery = await myContext.Test.FirstOrDefaultAsync(); // This will throw an exception
app.Run();
public class MyContext : DbContext
{
public MyContext() { }
public MyContext(DbContextOptions<MyContext> options) : base(options) { }
public virtual DbSet<TestSet> Test { get; set; }
}
public class TestSet
{
public Guid Id { get; set; }
public FunnyEnum FunnyEnum { get; set; }
}
public enum FunnyEnum
{
Yes
}
csproj is just web sdk and npgsql efcore
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net9.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="9.0.1" />
</ItemGroup>
</Project>
Facing the same issue, it seems like the MapEnum options write to a options.EnumDefinitions
and options is a singleton INpgsqlSingletonOptions so calling UseNpgsql twice ends up adding the enum twice to the list :/
In my config I'm reusing the same exact npgsql options - I get the same one via a method
services.AddDbContext<DbContext>((servicesProvider, options) => DbContextOptionBuilder(options, servicesProvider), ServiceLifetime.Transient);
services.AddPooledDbContextFactory<DbContext>((servicesProvider, options) => DbContextOptionBuilder(options, servicesProvider));
Sadly the old way to do enums don't seems to work anymore and this cause issues with the new way.
Is there anyway to load the config somehow outside of the dbcontext adds ?
something like
service.AddSingleton<INpgsqlSingletonOptions>(new NpgsqlSingletonOptionsBuilder().TheOptions());
services.AddDbContext<DbContext>(ServiceLifetime.Transient);
services.AddPooledDbContextFactory<DbContext>();
Thanks for the minimal repro @LucHeart - I can see the problem happening.
So the way EF configuration works, if you specify both AddDbContextPool() and AddPooledDbContextFactory(), the configuration lambdas in both calls incrementally configure the same options; the context instances you get from both injection methods (direct injection, injection of the context factory) have the same options and behave identically.
This means that your code is effectively the same as duplicate invocation of MapEnum, i.e.:
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
=> optionsBuilder
.UseNpgsql("Host=localhost;Username=test;Password=test", o =>
{
o.MapEnum<FunnyEnum>();
o.MapEnum<FunnyEnum>();
})
I'll think a bit and consult with other EF members to see what we think of this... I could of course add logic to the PG provider that detects duplicate and/or conflicting mapping (keep in mind that the 2nd MapEnum could specify a different store type, or name translator - that would be incompatible and have to throw).
But it seems wrong for users to duplicate the context configuration just because the want to use both AddDbContextPool() and AddPooledDbContextFactory(). So at least as a temporary workaround, I'd recommend doing the following:
services.AddDbContextPool<MyContext>(optionsBuilder =>
optionsBuilder.UseNpgsql(connectionString, npgsql => npgsql.MapEnum<FunnyEnum>()));
services.AddPooledDbContextFactory<MyContext>(_ => {});
In other words, configure only once - in the first lambda - and do nothing in the second. But as I wrote above, I'll consult with the EF team to understand this better.
/cc @ajcvickers
Minimal repro without ASP.NET and some changes
var connectionString = "Host=localhost;Username=test;Password=test";
var services = new ServiceCollection();
services.AddDbContextPool<MyContext>(optionsBuilder =>
optionsBuilder.UseNpgsql(connectionString, npgsql => npgsql.MapEnum<FunnyEnum>()));
// services.AddPooledDbContextFactory<MyContext>(optionsBuilder =>
// optionsBuilder.UseNpgsql(connectionString, npgsql
// => npgsql.MapEnum<FunnyEnum>()));
services.AddPooledDbContextFactory<MyContext>(_ => {});
var serviceProvider = services.BuildServiceProvider();
using var scope = serviceProvider.CreateScope();
var contextNotFromFactory = scope.ServiceProvider.GetRequiredService<MyContext>();
var contextFactory = scope.ServiceProvider.GetRequiredService<IDbContextFactory<MyContext>>();
using var contextFromFactory = contextFactory.CreateDbContext();
await contextNotFromFactory.Database.EnsureDeletedAsync();
await contextNotFromFactory.Database.EnsureCreatedAsync();
_ = await contextNotFromFactory.Test.FirstOrDefaultAsync();
_ = await contextFromFactory.Test.FirstOrDefaultAsync();
public class MyContext : DbContext
{
public MyContext() { }
public MyContext(DbContextOptions<MyContext> options) : base(options) { }
public virtual DbSet<TestSet> Test { get; set; }
}
public class TestSet
{
public Guid Id { get; set; }
public FunnyEnum FunnyEnum { get; set; }
}
public enum FunnyEnum { Yes }
From discussion with the EF team: we agree that double configuration is a bad thing (i.e. repeating the same MapEnum twice); we may look into allowing both DbContext and IDbContextFactory to be registered in DI via a single call (https://github.com/dotnet/efcore/issues/26528).
Regardless, we think it's a good idea for the PG provider to handle this scenario better, i.e. detect and ignore duplicate MapEnum invocations, and throw (or possibly do last-one-wins) for incompatible invocations.
I agree..
In other words, configure only once - in the first lambda - and do nothing in the second
thats what I've been doing in the meantime, just looks kinda sketchy but works fine
FWIW I think repeating the same configuration twice is even more sketchy... :/
What is the status on this? im also runnign into this, but we are not using AddDbContextPool nor AddPooledDbContextFactory, we are using
_contextOptions = new DbContextOptionsBuilder<DbContext>()
.UseNpgsql(_dataSource, builder =>
{
builder.UseNodaTime();
builder.RegisterEnums(); // This calls the various builder.MapEnum<> types we have
})
.Options;
And we re-use those _contextOptions every time we need a new context:
public DbContext CreateContext()
{
return new DbContext(_contextOptions ?? throw new InvalidOperationException("PostgresFixture was not initialized"));
}
Then when configuring DI:
serviceCollection.AddDbContext<DbContext>(options
=> options.UseNpgsql(_dataSource, builder =>
{
builder.UseNodaTime();
builder.RegisterEnums(); // This calls the various builder.MapEnum<> types we have
});
@MJLHThomassenHadrian I'm not too clear on what exactly you're doing, why you're configuring your options twice (once in DI and once outside?), and whether this is really related to this issue. At minimum, please post a minimal, runnable repro.
Yeah, im sorry, its not really clear to me what we are doing. The codebase is pretty large and the UseNpgsql are also called in other extension methods so its not straightforward to replicate. Im just getting the same error as here so i though it was related. Im going to try and create a minimal example and post back.
While your issue is related to this, since you are facing the same issue essentially, it doesnt add much value here. Using the same DbContextOptionsBuilder or even the same NpgsqlDataSourceBuilder doesnt resolve this issue - would be the key takeaway for me here. It behaves the exact same as if you were just mapping the enums within each AddDbContext call. I already knew this but didnt include an example.
I agree, i finally figured out what we where doing btw.
In our integration tests which use WebApplicationFactory for building our app, we use a Postgresql Testcontainer. So during DI setup of our test, we are overwriting the regular DbContext registration with the test one. Something along the lines of:
_webAppFactory = webAppFactory.WithWebHostBuilder(builder =>
{
builder.UseTestServer().ConfigureTestServices(services =>
{
services.Remove<DbContextOptions<MyDbContext>>();
services.Remove<MyDbContext>();
services.Remove<NpgsqlDataSource>();
services.AddDbContext<MyDbContext>(options =>
{
options.UseNpgsql(dataSource, options => MapEnums(options));
});
//... other configuration
});
});
Where MapEnums maps all the enums for us.
The call to MapEnums here duplicates what the app's DI already has done (it also calls services.AddDbContext<MyDbContext>) and since apparantly these enums mapings are static or something, duplicates get registered.
For me the solution was to remove the enum mapings in the test's WebApplicationFactory registration.