Configure Logger headers declaratively with JSON

[frank-ang]

Summary

Opentelemetry sink incorrectly ignores request headers when the Logger is configured declaratively using JSON ConfigurationBuilder. In contrast, request headers are sent correctly only when the Logger is configured programmatically.

Steps to reproduce

1. Configure a header section of Serilog.Sinks.OpenTelemetry sink in the JSON configuration file.

    "WriteTo": [
      {
        "Name": "OpenTelemetry",
        "Args": { 
          "endpoint": "http://127.0.0.1:4318/v1/logs",
          "protocol": "HttpProtobuf",
          "headers": {
            "Authorization": "Bearer TOKEN_REDACTED"          
          }
        },
      }
    ]

2. Build the logger using JSON

var configuration = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json")
    .Build();
Log.Logger = new LoggerConfiguration()
    .ReadFrom.Configuration(configuration)
    .CreateLogger();

3. Call the logger, Log.Information()

Expected Result.

• Authentication succeeds.
• Network packet capture should show headers are included in requests to OpenTelemetry endpoint.

Actual Result.

• Authentication fails.
  • Exception when using HttpProtobuf protocol: Exception while emitting periodic batch from Serilog.Sinks.PeriodicBatching.PeriodicBatchingSink: System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
  • Exception when using Grpc protocol: Exception while emitting periodic batch from Serilog.Sinks.PeriodicBatching.PeriodicBatchingSink: Grpc.Core.RpcException: Status(StatusCode="Unauthenticated",
• Network packet capture (Wireshark) verifies headers are missing from requests to OpenTelemetry endpoint.

Potentially Related

• https://github.com/serilog/serilog-sinks-opentelemetry/issues/107
• https://github.com/serilog/serilog-settings-configuration/issues/398
• Headers, like ResourceAttributes, is an IDictionary data type https://github.com/serilog/serilog-sinks-opentelemetry/blob/3b04262e0f15e0571c1015772795a7d482a2eb86/src/Serilog.Sinks.OpenTelemetry/Sinks/OpenTelemetry/OpenTelemetrySinkOptions.cs#L63