ApplicationInsights-dotnet icon indicating copy to clipboard operation
ApplicationInsights-dotnet copied to clipboard

DefaultAggregationPeriodCycle threads are not stopped

Open avereshchak opened this issue 4 years ago • 5 comments

  • List of NuGet packages and version that you are using:
    • Microsoft.ApplicationInsights.WorkerService 2.15.0
  • Runtime version: net472
  • Hosting environment: Windows Console application

What are you trying to achieve? We would like to switch between different telemetry providers without restarting server side applications. For example, a sysadmin may decide to temporarily enable AppInsights telemetry, then switching it off after some period of time and use in-house telemetry implementation.

I've discovered that after disposing the container which was configured to host the AppInsights Worker Service, the DefaultAggregationPeriodCycle threads are left in Running state. If AppInsights Worker Service is initialized again, new worker threads will be spawned.

What have you tried so far? I've tried disposing the DI container itself which was hosting the AppInsights Worker Service, tried TelemetryConfiguration.Disable = true, disposed TelemetryConfiguration - no luck, the threads remain alive.

After some analysis I've found that DefaultAggregationPeriodCycle initiates worker thread shutdown:

  1. When a redundant MetricManager is created due to a racing condition.
  2. From a DefaultAggregationPeriodCycle's finalizer (https://github.com/microsoft/ApplicationInsights-dotnet/blob/7633ae849edc826a8547745b6bf9f3174715d4bd/BASE/src/Microsoft.ApplicationInsights/Metrics/Implementation/DefaultAggregationPeriodCycle.cs#L41).
  3. From a MetricManager's finalizer.

However it appears the finalizers in 2) and 3) are never called because:

  • both DefaultAggregationPeriodCycle and MetricManager are referencing each other
  • the worker thread holds a reference to DefaultAggregationPeriodCycle.runningState field (https://github.com/microsoft/ApplicationInsights-dotnet/blob/7633ae849edc826a8547745b6bf9f3174715d4bd/BASE/src/Microsoft.ApplicationInsights/Metrics/Implementation/DefaultAggregationPeriodCycle.cs#L154)

Is this a bug or there is a legal way for shutting these threads down gracefully?

For your convenience I'm attaching a sample project which reproduces the problem.

Thanks!

avereshchak avatar Oct 07 '20 16:10 avereshchak

And here comes an even simpler example reproducing the problem:

using System;
using System.Threading;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.ApplicationInsights.Metrics;

namespace AppInsightsThreadLeak
{
    class Program
    {
        static void Main()
        {
            UseAppInsights();

            Thread.Sleep(5000);
            
            GC.Collect();
            GC.WaitForPendingFinalizers();
            GC.Collect();

            Console.WriteLine("Attach with debugger and see if DefaultAggregationPeriodCycle thread(s) are still alive.");
            Console.ReadLine();
        }

        private static void UseAppInsights()
        {
            // Real instrumentation key isn't needed.
            var configuration = new TelemetryConfiguration("");
            var client = new TelemetryClient(configuration);

            // Internally a background thread is started by DefaultAggregationPeriodCycle.
            var metricManager = configuration.GetMetricManager();

            // Now try our best to stop the telemetry.
            configuration.DisableTelemetry = true;
            configuration.TelemetryChannel.Flush();
            configuration.Dispose();
        }
    }
}

avereshchak avatar Oct 08 '20 09:10 avereshchak

This issue is stale because it has been open 300 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] avatar Sep 15 '21 00:09 github-actions[bot]

The problem has not been fixed yet.

avereshchak avatar Sep 17 '21 04:09 avereshchak

This issue is stale because it has been open 300 days with no activity. Remove stale label or this will be closed in 7 days. Commenting will instruct the bot to automatically remove the label.

github-actions[bot] avatar Jul 15 '22 00:07 github-actions[bot]

Still valid.

avereshchak avatar Jul 15 '22 08:07 avereshchak

This issue is stale because it has been open 300 days with no activity. Remove stale label or this will be closed in 7 days. Commenting will instruct the bot to automatically remove the label.

github-actions[bot] avatar May 13 '23 00:05 github-actions[bot]

Is there any chances getting this issue addressed?

avereshchak avatar Aug 15 '23 08:08 avereshchak