[Feedback Request] Logging complex types in .NET

[CodeBlanch]

Greetings OpenTelemetry .NET community!

We are working with the dotnet/runtime team to explore expanding the ILogger .NET API to support passing complex types through the pipeline. The goal is to allow ILoggerProvider implementations (such as OpenTelemetry) to walk the public properties of a payload passed to the logger and then export that data into log streams.

I am looking to gather some customer use cases to present/validate in the design discussions. If you have experience in this area please share some details about your experience, needs, goals, challenges, etc.

Some questions, but these are not exhaustive or mandatory to be answered:

• How are you logging complex types today? ILogger? Some other logging library?
• Is masking/redaction a concern? How are you handling this today?
• What types of things are being logged? Classes, structs, third-party objects?
• What kind of performance goals do you have?
• How do you use your logging data after it is collected?

Relates to https://github.com/dotnet/runtime/issues/61947

[mahaisong]

At present, there are few plug-in types of opentelemetry. Only SqlClient is easy to use. You can refer to skyapm Net has many plug-ins with complete functions, and the messages that can be subscribed based on diagnostic are output.

[cijothomas]

At present, there are few plug-in types of opentelemetry. Only SqlClient is easy to use. You can refer to skyapm Net has many plug-ins with complete functions, and the messages that can be subscribed based on diagnostic are output.

It is unclear how is this related to this issue? Could you clarify?

[p10tyr]

How are you logging complex types today? ILogger? Some other logging library?

We only use ILogger and if we need to export logs to anything we write our own LoggerProviders to wrap 3rd party implementation.

Is masking/redaction a concern? How are you handling this today?

It is a high concern that is why we do not allow Complex Types to be serialised or even passed in. All our log messages are formatted strings that go through Peer Review to avoid PII leaks

What types of things are being logged? Classes, structs, third-party objects?

None. Just hand cranked logs strings as needed

What kind of performance goals do you have?

NA

How do you use your logging data after it is collected?

Support and Analytics

---

One thing that is very important for us is to be able to group our log data by tenant (transparently on every single log or trace). This is proving to be difficult to implement with the current OTLP Logger because there is no way to add our own attributes.

Please reference https://github.com/open-telemetry/opentelemetry-dotnet/discussions/3275

--Edit Solved - Apologies for hijacking this thread a bit. Thanks for the help and all the work gone into OTLP!!!

[bnjkukunuri]

Currently, is there a way to provide custom formatter (Func<TState, Exception?, string> formatter) to log in opentelemetry logger, is there a recommended pattern? I am trying to have a Log processor to redact value of any property that is decorated with certain attribute so that logs does not show the actual sensitive data. Log Processor has access to LogRecord.StateValues and have the necessary redaction logic. however the formatted message in LogRecord has the original value. This seems because of issue

I see that formatter in OpenTelemetryLogger is based on object.ToString() Is there any plan to provide a way to provide a custom formatter to OpenTelemetryLogger?

[zyofeng]

We currently utilize Serilog and Destructurama.Attributed to mask log data on a property level It would be great to have something similar.

[xakep139]

Currently, is there a way to provide custom formatter (Func<TState, Exception?, string> formatter) to log in opentelemetry logger, is there a recommended pattern? I am trying to have a Log processor to redact value of any property that is decorated with certain attribute so that logs does not show the actual sensitive data. Log Processor has access to LogRecord.StateValues and have the necessary redaction logic. however the formatted message in LogRecord has the original value. This seems because of issue

I see that formatter in OpenTelemetryLogger is based on object.ToString() Is there any plan to provide a way to provide a custom formatter to OpenTelemetryLogger?

You can use ILogger.Log() with any ILogger and pass any custom formatter.

[amccool]

• How are you logging complex types today? ILogger? Some other logging library? ---> ILogger. Exception is the only complex data type. sometimes we serilize to json and pump that string. However that is disliked
• Is masking/redaction a concern? How are you handling this today? ---> redaction is done with governance.
• What types of things are being logged? Classes, structs, third-party objects? ---> see complex types. the only complex type is exception
• What kind of performance goals do you have? --> moving to LoggerMessage deletes in net6. offload all the write to the collector
• How do you use your logging data after it is collected? ---> TDB, currently a custom ilogger to elasticsearch https://www.nuget.org/packages/AM.Extensions.Logging.ElasticSearch

[cijothomas]

I think we can close, given .NET supports this now, and is documented here : https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/docs/logs/complex-objects