404 on all static files as soon as authorization is required

I have a very strange behavior. As soon as I am not on "Development" I get 404s on all static files

I added app.UseCoreAdminCustomAuth((serviceProvider) => Task.FromResult(true)); to make sure I am always authenticated. But still, the issue happens. my program.cs looks like this:

var builder = WebApplication.CreateBuilder(args);

builder.Configuration.AddJsonFile("appsettings.local.json", optional: true, reloadOnChange: true);
builder.Configuration.AddEnvironmentVariables();

builder.Services.AddHttpContextAccessor();
builder.Services.AddServices();
builder.Services.AddControllers();

builder.Services.AddDbContext<MyDbContext>(option =>
    option.UseSqlServer(builder.Configuration.GetConnectionString("DefaultConnection")));

// add core admin
builder.Services.AddCoreAdmin();

var app = builder.Build();

app.UseStaticFiles();
app.UseAuthorization();
app.MapControllers();
app.MapDefaultControllerRoute();
app.UseCoreAdminCustomAuth((serviceProvider) => Task.FromResult(true));
app.Run();

Did you ever find a solution, this is very frustrating running any env other then Development seems to cause this