Object disposed exception while running unit tests when accessing ServiceProvider in filter.

[voroninp]

I am running ASP.NET Core unit tests. Test class implements IDisposable interface and disposes WebApplicationFactory.

Unfortunately, I am getting exception in my HF's filter which tries to create a new scope for services. It looks like HF server is not stopped properly when Host is stopping.

Here's how I register HF and filters:

services.AddHangfire((provider, configuration) =>
{
    if (_configuration.GetValue<bool>("HangfireConfiguration:InMemory"))
    {
        configuration.UseMemoryStorage();
    }
    else
    {
        configuration.UsePostgreSqlStorage(_configuration.GetConnectionString("OrderFulfillment"));
    }
    configuration.UseFilter(
        new HangfireExceptionLoggingFilter(
            provider.GetRequiredService<ILogger<HangfireExceptionLoggingFilter>>()));
    configuration.UseFilter(
        new OutboxableJobsFilter<OrderFulfillmentDbContext>(provider));
});

and here's the code of the filter:

public sealed class OutboxableJobsFilter<TDbContext> : IServerFilter
    where TDbContext : DbContext
{
    private readonly IServiceProvider _provider;

    public OutboxableJobsFilter(IServiceProvider provider)
    {
        _provider = provider.NotNull();
    }

    public void OnPerformed(PerformedContext context)
    {
        var isNotOutboxable =
            !context.BackgroundJob.Job.Type.IsAssignableTo(typeof(IOutboxableJob))
            && context.BackgroundJob.Job.Type.GetCustomAttribute<OutboxableJobAttribute>() is null
            && context.BackgroundJob.Job.Method.GetCustomAttribute<OutboxableJobAttribute>() is null;
        if (isNotOutboxable)
        {
            return;
        }

        if (context.Canceled || context.Exception is not null)
        {
            return;
        }

        TryToRemoveJobFromOutboxedJobsList(context);
    }

    /// <summary>
    /// Tries to perform a cleanup action. If something fails during this operation,
    /// exception will be logged and swallowed. At the moment of running this operation job
    /// has completed successfully, and it is better to avoid retrials.
    /// </summary>
    private void TryToRemoveJobFromOutboxedJobsList(PerformedContext context)
    {
        try
        {
            using var scope = _provider.CreateScope();
            var dbContext = scope.ServiceProvider.GetRequiredService<TDbContext>();
            var jobId = context.BackgroundJob.Id;
            var outboxedJob = new OutboxedJob(jobId);
            dbContext.Set<OutboxedJob>().Remove(outboxedJob);
            // This will throw, if there's no outbox job, but it indicates some bug.
            // We should not be here then.
            dbContext.SaveChanges();
        }
        catch (Exception ex)
        {
            var logger = _provider.GetRequiredService<ILogger<OutboxableJobsFilter<TDbContext>>>();
            logger.LogError(ex, "Error when removing job from the list of Outboxed Jobs.");
        }
    }

    public void OnPerforming(PerformingContext context)
    {
        var isNotOutboxable =
            !context.BackgroundJob.Job.Type.IsAssignableTo(typeof(IOutboxableJob))
            && context.BackgroundJob.Job.Type.GetCustomAttribute<OutboxableJobAttribute>() is null
            && context.BackgroundJob.Job.Method.GetCustomAttribute<OutboxableJobAttribute>() is null;
        if (isNotOutboxable)
        {
            return;
        }

        if (context.Canceled)
        {
            return;
        }

        CancelExecutionIfOutboxedJobsListDoesNotContainJob(context);
    }

    private void CancelExecutionIfOutboxedJobsListDoesNotContainJob(PerformingContext context)
    {
        var scope = _provider.CreateScope();
        var dbContext = scope.ServiceProvider.GetRequiredService<TDbContext>();
        var jobId = context.BackgroundJob.Id;
        var outboxedJobExists = dbContext.Set<OutboxedJob>()
            .Any(x => x.JobId == jobId);

        if (!outboxedJobExists)
        {
            // The execution of outbox job is meaningful only in case of successful transaction
            // which lead to scheduling the job.
            // When transaction fails, there won't be any record, so it indicates, that job must me canceled.
            context.Canceled = true;
        }
    }
}

This filter is just a trick for implementing extended variant of Transactional Outbox pattern.

We've discovered the issue when we wrapped the incoming API call into HF job for async processing. We had to introduce a delay of 30 seconds in each API test, so everything is properly cleaned up. I played with polling intervals, but they have no effect within this tame range.

[odinserj]

Could you please also send full details for the ObjectDisposedException you have, including its stack trace?

[voroninp]

Ok, I'll try to reproduce, and then share the details.

[lloydkevin]

How about using a wrapper class, inject the IServiceProvider in there and use that to resolve the OrderFulfillmentDbContext?