Sporadic TypeLoadException

[mrtristan]

We're randomly getting the following exception in our Dockerized net7.0 apps. We have a few dozen different apps that are instrumented the same way and running in the same microsoft-provided runtime images, we've only seen it happen twice so far (once each in two different apps).

System.TypeLoadException: Could not load type 'Datadog.Trace.ClrProfiler.CallTarget.CallTargetInvoker' from assembly 'REDACTED, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.

The issue is resolved by just relaunching the (AWS ECS Fargate) app, not sure yet how to reproduce.

using 2.42.0 of nuget package for potentially setting tags and also have the tracer manually installed (same version) in the dockerfile

relevant bits of the dockerfile:

[andrewlock]

Hi @mrtristan, sorry you're having issues! Would it be possible to share the tracer logs from these instances? Ideally we'd like to get Debug logs (DD_TRACE_DEBUG=1) but given it only happens sporadically, I understand that could be tricky, so any logs you could get would be great.

Just to clarify, is the REDACTED in the message above one of your own libraries? Is it the main dotnet application that you're running?

I assume from your message that both the installed (automatic) and Datadog.Trace NuGet (manual) are the same version, 2.42.0?

Thanks!

[mrtristan]

both the same version, yes. REDACTED is the main application.

I'd be happy to share the logs but the opportunity to grab them is likely gone. i had to reset the running task in AWS to bring it back up which disposes of that entire runtime. i have tracer logs captured in cloudwatch but they're just info level it seems, no errors in there.

here's a thought though, maybe silly. could a StackOverflow from elsewhere yield this in any way? there was a serilog bug that was just recently patched that would yield a SO and i'm wondering if that could then cause this in some fashion? ref: https://github.com/serilog/serilog/releases/tag/v3.1.1

[andrewlock]

i have tracer logs captured in cloudwatch but they're just info level it seems, no errors in there.

Those could still be interesting, my vague hypothesis is that the initial injection is incorrect, and there may be something in the native tracer logs. If you could open a ticket here with those logs (and mention that you have an issue open here, and that we requested you submit the logs) then we'll look into it asap!

could a StackOverflow from elsewhere yield this in any way

I think it's unlikely, but would need to dig in further.

One more question - how are you publishing your app? Are you using framework dependent or self-contained? Are you using "publish single file"? Are you using trimming? Just some more data points for us to take into consideration 🙂

Thanks!

[mrtristan]

just a restore and then RUN dotnet publish -c Release -o out --runtime linux-x64 --self-contained false

no trimming either

and just to reiterate, everything works perfectly across logs/apm/asm/etc once relaunched.

will send those logs in shortly.