.net6 minimal API test config overrides only apply after app has been built

[maisiesadler]

Describe the bug

Using .net 6 minimal APIs and WebApplicationFactory for testing, the test config override is only applied after .Build. This means any interaction with config while configuring services using builder.Configuration will be using the app configuration, not test overrides.

To Reproduce

I've created a repo that shows the difference, here is the Program, and an Example test.

Test run

Further technical details

• ASP.NET Core version: net6.0

[martincostello]

I think the problem here is that you're binding the configuration into your service while the application is still in its bootstrapping phase (which is until Build()), so the configuration isn't yet "done". It's similar to the sort of issue where you need configuration to configure other things, like if you're say using Azure Key Vault, and need to use config to configure the Key Vault provider itself.

Does it work as you'd expect if you change it to this instead?

builder.Services.AddSingleton(() => new Options(builder.Configuration["AppSettings:Options"]));

I think if you resolve it lazily like that, when the lambda to get Options runs the Configuration would then have the extra settings in it from the tests.

[maisiesadler]

Hey Martin 👋

Yep that works - this might just be a gotcha rather than a bug since the Configuration is available on builder but it's not "Done" :( It also works using .net 6 but WebHost.CreateDefaultBuilder rather than the new WebApplication.CreateBuilder diff test run

[martincostello]

Yeah it's probably more of a gotcha of the new model - the WebApplicationFactory code gets invoked as part of a callback that gets called before Build() returns back to the Program class, so the code that adds your test settings won't even have run yet when you register that dependency so it can't see your overrides as they don't exist yet.

[maisiesadler]

Makes sense. Will update the code and close the issue - thanks!

[halter73]

Reopening so we document this. This might also be worth trying to fix in .NET 7. Thanks for the great repro.

[davidfowl]

@halter73 are you gonna doc this?

[kristofferjalen]

Here's my question on SO on the same issue: https://stackoverflow.com/questions/69986598/with-webapplicationfactory-add-configuration-source-before-program-cs-executes

[twinter-amosfivesix]

I stumbled upon a workaround for this issue.

If you override WebApplicationFactory<T>.CreateHost() and call IHostBuilder.ConfigureHostConfiguration() before calling base.CreateHost() the configuration you add will be visible between WebApplication.CreateBuilder() and builder.Build(). From my testing it's visible in all the various places you'd look at the config (callbacks, etc.).

I have not tried it with adding a JSON file but I imagine it should work.

Note that calling IHostBuilder.ConfigureAppConfiguration() in this manner does not work. DeferredHostBuilder treats host configuration differently from app configuration. (Code part 1 - Code part 2)

The comments in DeferredHostBuilder.ConfigureHostConfiguration actually say "We can do this for app configuration as well if it becomes necessary." Perhaps it's necessary now. 😄

(The PR that made ConfigureHostConfiguration() work like this in DeferredHostBuilder.)

Example Program

var builder = WebApplication.CreateBuilder(args);

var result = builder.Configuration["Result"];

var app = builder.Build();

app.MapGet("/", () => result);

app.Run();

public partial class Program{ }

Example appsettings.json

{
    "Result" : "Normal Result"
}

Example Test

using System.Collections.Generic;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using NUnit.Framework;

namespace TestProject;

[TestFixture]
public class ProgramFixture
{
    [Test]
    public async Task Should_return_override()
    {
        var factory = new MyWebApplicationFactory();

        var client = factory.CreateClient();

        var result = await client.GetStringAsync("/");

        Assert.That(result, Is.EqualTo("Override"));
    }

    private class MyWebApplicationFactory : WebApplicationFactory<Program>
    {
        protected override IHost CreateHost(IHostBuilder builder)
        {
            builder.ConfigureHostConfiguration(config =>
            {
                config.AddInMemoryCollection(new Dictionary<string, string> { { "Result", "Override" } });
            });

            // This does not work.
            //builder.ConfigureAppConfiguration(config =>
            //{
            //    config.AddInMemoryCollection(new Dictionary<string, string> { { "Result", "Override" } });
            //});

            return base.CreateHost(builder);
        }

        // This does not work.
        //protected override void ConfigureWebHost(IWebHostBuilder builder)
        //{
        //    builder.ConfigureAppConfiguration(config =>
        //    {
        //        config.AddInMemoryCollection(new Dictionary<string, string> { { "Result", "Override" } });
        //    });
        //}
    }
}

[bvachon]

The above workaround did not work for me because I wanted the AddInMemoryCollection to override the existing config (ie: be added last). So here's what I came up with instead.

Program.cs:

    var builder = WebApplication.CreateBuilder(args);

    builder.WebHost.ConfigureAppConfiguration(builder =>
    {
        builder
            .AddInMemoryCollection(s_ConfigOverride);
    });
    
    ...

    public partial class Program 
    {
        static Dictionary<string, string> s_ConfigOverride { get; set; } = new();
    
        class ClearConfigOverride : IDisposable
        {
            public void Dispose() => s_ConfigOverride = new Dictionary<string, string>();
        }
    
        public static IDisposable OverrideConfig(Dictionary<string, string> config)
        {
            s_ConfigOverride = config;
            return new ClearConfigOverride();
        }
    }

UnitTest.cs

      [OneTimeSetUp]
      public void OneTimeSetup()
      {
          using var _ = Program.OverrideConfig(new Dictionary<string, string>()
          {
              { "MyConfig", "someValue" }
          });

          var application = new WebApplicationFactory<Program>()
              .WithWebHostBuilder(builder =>
              {
                  // ... Configure test services
              });
        }
        ....

[diegosasw]

Having the same issue. I'm able to set the environment to "Test" for my functional tests but when reading settings at runtime (during tests), the appsettings.Test.json is being ignored when using minimal API

I use it like this:`

        var application =
            new WebApplicationFactory<Program>()
                .WithWebHostBuilder(
                    config =>
                        config
                            .UseCommonConfiguration()
                            .UseEnvironment("Test")
                            .ConfigureTestServices(ConfigureTestServices));

And I have the extension

public static class WebHostBuilderExtensions
{
    public static IWebHostBuilder UseCommonConfiguration(this IWebHostBuilder builder)
    {
        builder.ConfigureAppConfiguration((hostingContext, config) =>
        {
            var env = hostingContext.HostingEnvironment;

            config
                .AddJsonFile("appsettings.json", optional: true, reloadOnChange: false)
                .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: false)
                .AddEnvironmentVariables();

            if (env.IsDevelopment())
            {
                var appAssembly = Assembly.Load(new AssemblyName(env.ApplicationName));
                config.AddUserSecrets(appAssembly, optional: true);
            }
        });

        return builder;
    }
}

I don't understand why some of you suggest this is a gotcha rather than a bug.

If I set the webhost builder to use environment Test, I would expect the IConfiguration to read from the appsettings.Test.json.

Also, why do you say it applies only after app has been built? I can see my builder.Build() happening before the IConfiguration is retrieved from the DI container and settings read

This is my Program.cs

using Rubiko.Logging.Microsoft.DependencyInjection;
using Sample.Extensions;

const string corsPolicy = "corsPolicy";

var builder = WebApplication.CreateBuilder(args);
var configuration = builder.Configuration;

// IoCC
var services = builder.Services;

services
    .AddMicrosoftFactoryLogging(configuration.GetSection("Logging"))
    .AddMicroserviceDependencies()
    .AddCors(corsPolicy);

var app = builder.Build();

// Http Pipeline
var assemblyPath = $"/{typeof(Program).Assembly.GetName().Name!.ToLowerInvariant()}";

app
    .UsePathBase(assemblyPath)
    .UseRouting()
    .UseCors(corsPolicy)
    .UseDeveloperExceptionPage()
    .UseOpenApi(assemblyPath)
    .UseAuthorization()
    .UseEndpoints(endpoints =>
    {
        endpoints.MapControllers();
    });

app.Run();

// See https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-6.0
public partial class Program { }

If anyone has a workaround, please let me know. But I will probably go back to the old fashioned Startup.cs and the

        var server =
            new TestServer(
                new WebHostBuilder()
                    .UseStartup<Startup>()
                    .UseCommonConfiguration()
                    .UseEnvironment("Test")
                    .ConfigureTestServices(ConfigureTestServices));

[humbe76]

This is blocking my effort to move to new ways of app init too.. Minimal code looks like a total anti-feature. To be able to show of a one-liner for hello world, we have to cope with a lot of new magic that complicates flexibility in unit testing, making it harder to make anything that is a real application..

Before .NET 6 we implement a main method, handle the command line arguments and call into web API init code. That way it is easy for us to make unit tests test everything and provide anything wanted from the unit test libraries into the application to mock, stub, and avoid running code that will blow up if not in production environment.

I'm unable to find any documentation of how to convert the minimal code to actual readable normal code which one can integrate with, and I have not been able to find a workaround to make a unit test to be able to provide input readable from the very beginning of the Program.cs file.

Going through a static variable of course works, but I want to avoid that as I want multiple unit tests to be able to run in parallel.

• Within the Program.cs one can not refer to this, so even using a static collection with this.GetHashCode() to index into it doesn't work.
• If I create members in a partial Program.cs implementation, I can't access them from the minimal code.
• There's no member in the auto-generated Program.cs file I can set something that is available at the top of Program.cs
• I've found no way to give it the command line arguments I want to test with.

All I've managed to do so far is inject a service that can be fetched after builder.Build() has been called, but that is too late..

[davidfowl]

We're going to tackle this in .NET 8, it'll require some new APIs to make it work well.

[kaylumah]

Inside protected override void ConfigureWebHost(IWebHostBuilder builder)

var config = new ConfigurationBuilder()
                .AddInMemoryCollection(new Dictionary<string, string>() {})
                .Build();
builder.UseConfiguration(config);

Appears to work as an alternative for the ConfigureAppConfiguration but not sure if that Is desired.

Guidance on this would be appreciated especially if it would be delayed to .NET8 Thanks Max

[mikeblakeuk]

Sad times.

[ghost]

Thanks for contacting us.

We're moving this issue to the .NET 8 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s). If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues. To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[opolkosergey]

I've faced with similar issue in my tests. I need to use class inherited from WebApplicationFactory<Program> in my tests. The issue is that in Production scenario Program.cs initializes Serilog with MSSqlServer sink. So I have to have to provide connectionString - it isn't what I would like to do in tests. Part of my appsettings.json (located in project under test) is: "Serilog": { "Using": [ "Serilog.Sinks.MSSqlServer" ], "MinimumLevel": { ... } }, "WriteTo": [ { "Name": "MSSqlServer", "Args": { "connectionString": "{connection_string}", "sinkOptionsSection": { ... }, "columnOptionsSection": { "customColumns": [ ... ] } } } ] }

This answer prompted me to find workaround for my case:

    internal class MyWebApplicationFactory : WebApplicationFactory<Program>
    {
        ...
        protected override IHost CreateHost(IHostBuilder builder)
        {
            builder.ConfigureHostConfiguration(config =>
            {
                config.AddJsonFile("appsettings.json");
            });
            return base.CreateHost(builder);
        }		
        ...
    }

where appsettings.json (located in test project) is:

{ "Serilog": { "WriteTo": [ { "Name": "Console" }, { "Name": "Console" } ] } }

Set this advanced settings for your test project

[JBastiaan]

So its really great that it is being worked on for .NET 8, but here we are on .NET 7. How is anyone supposed to make test configuration overrides work now? Are we just supposed to abandon the minimal hosting model until .NET 8? None of the above workaround work for me since im calling some extension method on startup that just requires a string value from config. Lazy loading won't help me, the overridden config value needs to be there when the app is starting. Did anyone find a workaround for this?

[diegosasw]

So its really great that it is being worked on for .NET 8, but here we are on .NET 7. How is anyone supposed to make test configuration overrides work now? Are we just supposed to abandon the minimal hosting model until .NET 8? None of the above workaround work for me since im calling some extension method on startup that just requires a string value from config. Lazy loading won't help me, the overridden config value needs to be there when the app is starting. Did anyone find a workaround for this?

I couldn't make it work either, so I abandoned minimal API and keep using the standard Startup + Program.

[cjbush]

So I was able to get this to work with a kind-of-hacky workaround. My Program.cs looks something like this:

var app = WebApplication.CreateBuilder(args)
                        .BuildDefaultConfiguration()
                        .ConfigureServices((services, configuration) => 
                        {
                            //configure IServiceCollection
                        })
                        .Build()
                        .ConfigureApplication(app => 
                        {
                            app.UseHealthChecks("/hc", new HealthCheckOptions
                            {
                                Predicate = _ => true,
                                ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
                            });

                            //define your other routes and what not
                        });
await app.RunAsync();

and I have the following extension methods defined:

public static class BootstrappingExtensions 
{
    public static WebApplicationBuilder BuildDefaultConfiguration(this WebApplicationBuilder builder)
    {
         if(builder.Configuration.GetValue("IsConfigured", false)) return builder; // this is the hacky/magic part
         //load appsettings.json or Azure App Config or whatever you want
         return builder;
    }
    

    public static WebApplicationBuilder ConfigureServices(this WebApplicationBuilder builder, Action<IServiceCollection, IConfiguration> configure)
    {
         configure(builder.Services, builder.Configuration);
         return builder;
    }

    public static WebApplication ConfigureApplication(this WebApplication app, Action<WebApplication> configure)
    {
         configure(app);
         return app;
    }
}

then in my test fixture I have:

[TestFixture]
public sealed class ProgramTest
{
    private sealed class ProgramFactory  : WebApplicationFactory<Program>
    {
        protected override IHost CreateHost(IHostBuilder builder)
        {
            builder.ConfigureHostConfiguration((configBuilder) => 
            {
                configBuilder.Sources.Clear();
                configBuilder.AddInMemoryCollection(new Dictionary<string,string>
                {
                    { "ConnectionStrings:MyDatabaseConnection", "<some test value>" },
                    { "IsConfigured", "true" }
                });
           });
        }
    }

    [Test]
    public async Task Program_HealthCheck_ShouldReturnHealthy()
    {
        using var application = new ProgramFactory();
        using var client = application.CreateClient();

        var response = await client.GetAsync("/hc");
    
        var report = await response.Content.ReadAsJsonAsync<UIHealthReport>(); //ymmv as far as json options goes here
        report.Should().NotBeNull().And.Match<UIHealthReport>(_ => _.Status == UIHealthStatus.Healthy);
    }
}

So the downside obviously is you're setting a test flag and checking for it in production code, but it does seem to work for overriding the config with minimal APIs until a real fix comes along in .NET 8.

[davidfowl]

OK seeing the responses here urged me to come up with something in the meantime while we build a cleaner solution to this. The solution has 2 parts to it. A line of code you have to add to the application when you want to override configuration before:

Application

var builder = WebApplication.CreateBuilder(args);

// This is the special line of code. It should be added in the place where you want to override configuration
builder.Configuration.AddTestConfiguration();

var result = builder.Configuration["Result"];

var app = builder.Build();

app.MapGet("/", () => result);

app.Run();

public partial class Program { }

Test project

using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.Configuration;

namespace TestProject1
{
    public class UnitTest1
    {
        [Fact]
        public async Task OverrideWithInMemoryConfiguration()
        {
            var waf = new WebApplicationFactory<Program>();

            TestConfiguration.Create(b => b.AddInMemoryCollection(new Dictionary<string, string?>
            {
                ["Result"] = "Hello World!"
            }));

            var client = waf.CreateClient();

            var response = await client.GetStringAsync("/");
            Assert.Equal("Hello World!", response);
        }
    }
}

Here's the magic that makes this work:

namespace Microsoft.Extensions.Configuration;

internal static class TestConfiguration
{
    // This async local is set in from tests and it flows to main
    internal static readonly AsyncLocal<Action<IConfigurationBuilder>?> _current = new();

    /// <summary>
    /// Adds the current test configuration to the application in the "right" place
    /// </summary>
    /// <param name="configurationBuilder">The configuration builder</param>
    /// <returns>The modified <see cref="IConfigurationBuilder"/></returns>
    public static IConfigurationBuilder AddTestConfiguration(this IConfigurationBuilder configurationBuilder)
    {
        if (_current.Value is { } configure)
        {
            configure(configurationBuilder);
        }

        return configurationBuilder;
    }

    /// <summary>
    /// Unit tests can use this to flow state to the main program and change configuration
    /// </summary>
    /// <param name="action"></param>
    public static void Create(Action<IConfigurationBuilder> action)
    {
        _current.Value = action;
    }
}

NOTE: This in the unit test, the call to Create must happen before the CreateClient() call on the WebApplicationFactory<T> . This is because the ExecutionContext needs to flow to the main application and that gets initialized during the first access to the client or services on the application.

EDIT: If you derive a class from WebApplicationFactory<T>, it's possible to setup configuration before the app runs:

public class UnitTest1
{
    [Fact]
    public async Task ImplicitOverride()
    {
        var app = new App();
        app.ConfigureConfiguration(b => b.AddInMemoryCollection(new Dictionary<string, string?>
        {
            ["Result"] = "Hello!"
        }));

        var client = app.CreateClient();

        var response = await client.GetStringAsync("/");
        Assert.Equal("Hello!", response);
    }
}

class App : WebApplicationFactory<Program>
{
    private Action<IConfigurationBuilder>? _action;

    public void ConfigureConfiguration(Action<IConfigurationBuilder> configure)
    {
        _action += configure;
    }

    protected override IWebHostBuilder? CreateWebHostBuilder()
    {
        if (_action is { } a)
        {
            // Set this so that the async context flows
            TestConfiguration.Create(a);
        }

        return base.CreateWebHostBuilder();
    }
}

This is a bit cleaner and lets you configure the WebApplicationFactory instead of a static.

[ayyron-dev]

Depending on the use-case, a less robust but fairly simple short term workaround could be to use Environment Variable configuration.

Environment.SetEnvironmentVariable("KEY", "VALUE");
var waf = new WebApplicationFactory<Program>();
var client = waf.CreateClient()
/* ... rest of test ... */

Edit: Don't do this, DavidFowl makes a really great point about how this could produce unexpected results.

[davidfowl]

Yea I don’t recommend that. It affects the entire process (including the test process). That’s no good, especially since tests run concurrently by default.

[diegosasw]

This seems to be the best workaround so far to override settings.

Also the .ConfigureServices seems to be executed AFTER the Program.cs registrations, so it's a good place to remove/replace service registrations.

With those two things it seems I can get by using mnimal API with .NET 7 without sacrificing integration tests or messing around with environment variables

Example:

protected IntegrationTestBase()
{
    // Workaround for overrides in .NET 7 and minimal API. See https://github.com/dotnet/aspnetcore/issues/37680
    var testConfigurationBuilder =
        new ConfigurationBuilder()
            .AddInMemoryCollection(
                new List<KeyValuePair<string, string?>>
                {
                    new("CoolSection:Colors:0", "red"),
                    new("CoolSection:Colors:1", "black"),
                })
            .Build();
    
    var webApplicationFactory =
        new WebApplicationFactory<Program>() // It requires public partial class Program { } at Program.cs in main assembly
            .WithWebHostBuilder(webHostBuilder =>
                webHostBuilder
                    .UseEnvironment("Development")
                    .UseConfiguration(testConfigurationBuilder)
                    .ConfigureServices(TestServiceOverrides));
    // ...
}

protected virtual void TestServiceOverrides(IServiceCollection servives)
{
     // Replace services in DI container. Virtual method so that test classes inheriting from IntegrationTestBase can override services also
}

Let's hope .NET 8 brings some good IWebHostedBuilder methods that work for these purposes. Because ConfigureTestServices and ConfigureAppConfiguration right now are misleading.

[MonocleKelso]

I just ran into this issue and wanted to add my 2 cents. It seems as if the behavior isn't consistent across different operating systems?

• builder.ConfigureAppConfiguration worked for me in Windows
• Had to switch to builder.ConfigureHostConfiguration for it to work in WSL (Ubuntu) - this also still worked in Windows
• builder.ConfigureHostConfiguration also worked when running tests in the mcr.microsoft.com/dotnet/sdk:7.0 Docker image but it wasn't loading the environment specific appsettings JSON config so the exceptions being thrown in WSL and Docker were different

[kolpav]

Anyone stumbling upon this issue, just rewrite your application to use Program.cs&Startup.cs, close this tab and be productive again, its that simple.

[davidfowl]

I’ll assign this one to myself

[Al4ric]

I have just discovered why any of the above was NOT working in my solution.

I was creating builder in my "Program.cs" like this: var builder = WebApplication.CreateBuilder();

I tried all of the above and in all different places (like ConfigureWebHost, CreateHost, etc.). Nothing worked.

Then I modified my Program.cs with this:

var builder = WebApplication.CreateBuilder(args);

And suddenly every method above was working. So by only adding args it is working (I guess it is identified as different method)

[dj-nitehawk]

I have just discovered why any of the above was NOT working in my solution.

I was creating builder in my "Program.cs" like this: var builder = WebApplication.CreateBuilder();

I tried all of the above and in all different places (like ConfigureWebHost, CreateHost, etc.). Nothing worked.

Then I modified my Program.cs with this:

var builder = WebApplication.CreateBuilder(args);

And suddenly every method above was working. So by only adding args it is working (I guess it is identified as different method)

omg thank you so much! i was losing my mind trying to hunt down why the environment set in WAF wasn't being picked up by the application. all i had to do was add the args to WebApplication.CreateBuilder method. so anyone reading this on .NET 7+ just make sure you create your app builder like this:

var builder = WebApplication.CreateBuilder(args);

[Rick-van-Dam]

Some ppl seem to imply/hope this is getting fixed in .NET 8. Can anyone confirm this is actually getting fixed in .NET 8? This is not just a gotcha but a breaking change in behavior between the old startup and the new WebApplication way of doing things. All the workarounds posted here seem to be pretty hacky and not a true solution to the problem.

[davidfowl]

It was not fixed in .NET 8.