opentelemetry-dotnet-contrib icon indicating copy to clipboard operation
opentelemetry-dotnet-contrib copied to clipboard

Add a Microsoft.Extensions.Logging Exporter

Open almostchristian opened this issue 4 years ago • 7 comments

The existing exporters in the main library don't play well with other Logging providers, such as Serilog, In particular, ConsoleExporter writes directly to System.Console, so its logs cannot be captured by Serilog. Also, Serilog does not play well with other logging providers so that in the example shown in the main library, where the OpenTelemetry logging provider was added, any logs written by this logging provider cannot be captured by any Serilog sink.

I propose an additional exporter which logs to the interfaces from the Microsoft.Extensions.Logging. This exporter replaces ConsoleExporter, since in a default aspnetcore application (or a console application using the GenericHost) includes a default logging provider, this exporter will log to Console. And since this exporter will use structured logging, logging providers such as Serilog will be able to extract the trace values and write them to a database for example.

The primary reason for wanting this library is for scenarios where you don't have a distributed tracing service such as Jaeger to export the trace logs to, and you want to use your existing logging providers as an export target.

https://github.com/open-telemetry/opentelemetry-dotnet/issues/2570

almostchristian avatar Nov 09 '21 01:11 almostchristian

I use serilog and this seemed to work for me: https://github.com/xavierjohn/OpenTelemetrySerilogSample/blob/main/OpenTelemetrySerilogSample/Program.cs

adamhathcock avatar Feb 11 '22 15:02 adamhathcock

I don't believe your code will work @adamhathcock when you configure Serilog with a sink other than the console. For example, I want to configure Serilog with the sqlserver sink to write to the db. If you add a file sink to your code, you will see that the OpenTelemetry logs you see in the console will not get logged to the file

using Serilog;
using OpenTelemetry.Logs;
using System.Reflection;
using Serilog.Events;

var builder = WebApplication.CreateBuilder(args);
builder.WebHost.UseSerilog((ctx, config) =>
{
    config
    .MinimumLevel.Information()
    .MinimumLevel.Override("Microsoft.AspNetCore", LogEventLevel.Warning)
    .Enrich.FromLogContext()
    .Enrich.WithProperty("Version", Assembly.GetExecutingAssembly().GetName().Version)
    .WriteTo.File("Logs/log.log");
}, writeToProviders: true);
builder.WebHost.ConfigureLogging((ctx, builder) =>
{
    builder.ClearProviders();
    builder.AddOpenTelemetry(options =>
    {
        options.AddConsoleExporter();
    });
});

almostchristian avatar Feb 12 '22 03:02 almostchristian

@almostchristian you could be correct. My legacy logs only to the console in a docker scenario to have an agent scrape those. Now i've added in the open telemetry stuff and it appears to work the same.

I haven't used a file sink or any other.

adamhathcock avatar Feb 14 '22 11:02 adamhathcock

A new exporter should be written to meet this requirement following the guide here : https://github.com/open-telemetry/opentelemetry-dotnet/tree/main/docs/trace/extending-the-sdk#exporter Exporter gets all traces, and it can in-turn emit it to ILogger.

cijothomas avatar Mar 02 '22 05:03 cijothomas

Should a mention be added that new metrics and traces exporters should be created (but not logs)? Exporting logs, which can already leverage ILogger, to another ILogger would probably not be that useful in 99% of cases.

julealgon avatar Jul 12 '22 22:07 julealgon

@almostchristian In your code above you are clearing the logger providers which removes Serilog's ILogger integration. Try removing that line:

builder.WebHost.ConfigureLogging((ctx, builder) =>
{
    // builder.ClearProviders(); <- Remove this
    builder.AddOpenTelemetry(options =>
    {
        options.AddConsoleExporter();
    });
});

CodeBlanch avatar Aug 01 '23 20:08 CodeBlanch

@almostchristian Did you ever get this to work? I am looking for exactly what you described.

giancarloaguilera avatar Feb 10 '24 02:02 giancarloaguilera

@adamhathcock: In the documentation of ConsoleExporter you will find the following warning:

This exporter is intended to be used during learning how telemetry data are created and exported. It is not recommended for any production environment.

That means in production you would never use the ConsoleExporter exporter. Instead I would say to use the OpenTelemetry.Exporter.OpenTelemetryProtocol exporter (see Link[1]) to send the log information to an OTEL collector or directly e.g. to an APM-Server of Elastic or to an endpoint who understands the Open Telemetry format.

@almostchristian: I don't see/understand the use case of this ticket. Technically if you use a exporter which logs information (in what format?) to the ILogger interface you have a recursive call. The only use case I see is to send the log information to the Console in an OpenTelemetry protocol (OTLP) format in JSON. This can be done also with an serilog formatter like Elastic.CommonSchema.Serilog for Elastic Common Scheme (ECS) to get the elastic native structure (see Link[2] and [3]). Unfortunately I haven't seen any serilog formatter for OTLP.

Link[1]: https://github.com/open-telemetry/opentelemetry-dotnet/ Link[2]: https://www.elastic.co/guide/en/ecs-logging/dotnet/current/serilog-formatter.html Link[3]: https://github.com/elastic/ecs-dotnet?tab=readme-ov-file#elasticcommonschemaserilog

HHobeck avatar Feb 20 '24 21:02 HHobeck