ApplicationInsights-dotnet icon indicating copy to clipboard operation
ApplicationInsights-dotnet copied to clipboard

TrackException does not send large Aggregate Exceptions to Application Insights

Open karpikpl opened this issue 10 months ago • 7 comments

  • List of NuGet packages and version that you are using:
    • "Microsoft.ApplicationInsights.AspNetCore": "2.22.0",
    • "Microsoft.ApplicationInsights.Kubernetes": "7.0.0",
  • Runtime version (e.g. net461, net48, netcoreapp2.1, netcoreapp3.1, etc. You can find this information from the *.csproj file):
    • <Project Sdk="Microsoft.NET.Sdk">
    • <OutputType>Exe</OutputType>
    • <TargetFramework>net8.0</TargetFramework>
  • Hosting environment (e.g. Azure Web App, App Service on Linux, Windows, Ubuntu, etc.):
    • Windows x64
    • Ubuntu 22

Describe the bug

I'm using following method to Track Exceptions:

	public void TrackException(Exception ex) {
		_telemetryClient.TrackException(new InvalidOperationException("Coordinator Crashed!"), _properties);
		// for some reason not all exceptions are sent to app insights with `TrackException`
		_telemetryClient.TrackTrace("Coordinator Crashed!: " + ex, SeverityLevel.Critical, _properties);
		_telemetryClient.TrackException(ex, _properties);
	}

I also tried with FlushAsync and Thread.Sleep/Task.Delay() to make sure telemetry is sent.

My service is throwing an exception when connection to Azure App Configuration using DefaultAzureCredentials.

Exception is:

System.TimeoutException: The provider timed out while attempting to load.
 ---> System.AggregateException: One or more errors occurred. (Service request failed.

With the reason for the error being the wrong scope for the token

The received access token does not have authorization to perform action over the requested scope or the scope is invalid. If access was recently granted, please refresh your credentials.

For some reason:

  1. _telemetryClient.TrackException(new InvalidOperationException("Coordinator Crashed!"), _properties); - WORKS
  2. _telemetryClient.TrackTrace("Coordinator Crashed!: " + ex, SeverityLevel.Critical, _properties); - WORKS
  3. _telemetryClient.TrackException(ex, _properties); - DOESN'T WORK - exception is nowhere to be found in app insights
  4. Using exception as inner exception doesn't help - it's not visible in app insights

Image

Exception is thrown when application is crashing (exiting). Adding Console.ReadLine() or flushing telemetry did not help.

[!WARNING] Is there anything in app insights that filters out that exception? it it related to it having WWW-Authenticate data?

To Reproduce

Sample app.

For Immediate Support

For immediate support relating to the Application Insights .NET SDK we encourage you to file an Azure Support Request with Microsoft Azure instead of filing a GitHub Issue in this repository. You can do so by going online to the Azure portal and submitting a support request. Access to subscription management and billing support is included with your Microsoft Azure subscription, and technical support is provided through one of the Azure Support Plans. For step-by-step guidance for the Azure portal, see How to create an Azure support request. Alternatively, you can create and manage your support tickets programmatically using the Azure Support ticket REST API

karpikpl avatar Jan 16 '25 21:01 karpikpl

not enough information to know why the Exception is getting dropped. You would need to collect the internal logs to see what issue the SDK is having. The easiest way to do that would be to follow the self-diagnostics method: https://github.com/microsoft/ApplicationInsights-dotnet/tree/main/troubleshooting/ETW#self-diagnostics

TimothyMothra avatar Jan 17 '25 01:01 TimothyMothra

does the log contains sensitive information? I can attach it here.

I've added

		} catch (Exception ex) {
			logging.TrackException(ex);
			await logging.FlushAsync(CancellationToken.None);
			Console.WriteLine(ex.Message, Color.Red);
			Task.Delay(5000).Wait();
			Console.WriteLine("Done waiting", Color.Red);
			}

but the log contains:

Successfully opened file. 2025-01-17T05:05:25.7310290Z:Process was called on the TelemetrySink after it was disposed, the telemetry data was dropped.{xxxApp}

Is it possible that Azure Credentials is too big? After ToString() it's 450 lines and 32803 characters long.

karpikpl avatar Jan 17 '25 05:01 karpikpl

This looks similar to #2284

karpikpl avatar Jan 17 '25 05:01 karpikpl

I've been able to reproduce here: https://github.com/karpikpl/app-insights-large-exception

karpikpl avatar Jan 17 '25 16:01 karpikpl

My guess is that SDK is not calculating "max" values when dealing with inner exceptions.

So when I have 1 large exception this code works fine https://github.com/microsoft/ApplicationInsights-dotnet/blob/c442ad30783d48b211650f7ba699735f175d6437/BASE/src/Microsoft.ApplicationInsights/Extensibility/Implementation/ExceptionConverter.cs#L15-L26

but when there are inner exceptions, each one of them is trimmed but in aggregate, they all exceed max payload size and are getting dropped.

karpikpl avatar Jan 17 '25 16:01 karpikpl

@TimothyMothra I could do a PR if you could help me identify where to make the changes

karpikpl avatar Jan 28 '25 15:01 karpikpl

I could work on a fix if given some direction, don't want to spend time if it's not getting merged :/

karpikpl avatar Apr 23 '25 00:04 karpikpl

I'm still interested in fixing this

karpikpl avatar Jul 25 '25 14:07 karpikpl