Log context conversion to tags

[adinauer]

In some languages log context is available (e.g. Java and .NET). Think of it as a Hashmap / Dictionary where you can store values which will then be printed with every emitted log entry.

In Java this is called MDC (Mapped Diagnostic Context). Here are some examples:

• https://www.slf4j.org/manual.html#mdc
• https://logback.qos.ch/manual/mdc.html
• https://logging.apache.org/log4j/2.x/manual/thread-context.html

Users have asked for context entries to be converted to Sentry tags (https://github.com/getsentry/sentry-java/issues/1148 and https://github.com/getsentry/sentry-java/issues/1054). This feature only existed in our .NET SDK. The config option there allows users to enable / disable the feature completely, but no fine grained control over which entries are converted to tags.

For Java we went with a bit more control over which entries will be picked up automatically from the MDC by adding a new config option called contextTags which can have a list of log context entry names (which have to match exactly, no glob / regex / similar for now). See this PR (https://github.com/getsentry/sentry-java/pull/1954) for more detail.

Our idea for future improvement on this feature (if needed) was to support globs in addition to exact entry names, e.g. myPrefix.* to automatically convert all entries starting with myPrefix. as Sentry tags. We can then also use this to have a convention of automatically converting all entries starting with sentry.tag. by making the default ["sentry.tag.*"]. This convention was also part of the old Java SDK (1.x) [TODO link].

When something similar is implemented, the default should be to not create too many tags automatically as they come at a cost.

[bruno-garcia]

Adding suggestions for Logging Library Integrations could be done under this heading: https://develop.sentry.dev/sdk/features/#http-client-integrations

There we can mention how to deal with tags.

We can also add other learnings, like mapping Scopes from logging libraries to Sentry scopes. That was done in the .NET SDK and turned out to have negative side effects. (S.E.L BeginScope -> Sentry PushScope) (e.g: https://github.com/getsentry/sentry-dotnet/issues/1492)

[kylejPomelo]

Hi @adinauer I was wondering if this was something you were still targeting, would be useful

[adinauer]

Hello @kylejPomelo, at the moment this isn't very high on our ToDo list. It would be great if you could share your use case with us so we can learn from it.

[kylejPomelo]

Hello @kylejPomelo, at the moment this isn't very high on our ToDo list. It would be great if you could share your use case with us so we can learn from it.

sound good, I understand.

Can't be too specific, but we store information that identifies a user in the MDC. We use the SentryAppender to send error-level logs to sentry.

I see the information in the sentry console under MDC when looking at an exception, but I'm unable to use that user-identifying field in a search, and have some difficulty searching errors per user as a result. I also have written a tool that will call the sentry API endpoint, and can't use the field as a query parameter (returns an empty list).

After looking a little closer at the source code, it seems I can define mdcProperties I want to track as context tags, and it will pick them up from the MDC for me? Sharing v6.1.4 of the appender since it's what I use in my code https://github.com/getsentry/sentry-java/blob/6.1.4/sentry-logback/src/main/java/io/sentry/logback/SentryAppender.java#L120-L134

I'll play around with that but I might be good for now, thanks

[adinauer]

@kylejPomelo thanks for sharing 🙏. Yes, contextTags is already available. I thought you were talking about the more advanced use cases mentioned in this issue. We haven't documented it yet (https://github.com/getsentry/sentry-java/issues/2351) but you can see how to use it in the samples (https://github.com/getsentry/sentry-java/blob/efa7b3ddfa946241933996ab5bddb8c3f7bb1642/sentry-samples/sentry-samples-logback/src/main/resources/logback.xml#L14-L15).

Is there any reason you're not upgrading to the latest version of the SDK? Is something broken for you?