ecs-logging-java icon indicating copy to clipboard operation
ecs-logging-java copied to clipboard

MDC values for event.type and event.category are not properly serialized as JSON arrays

Open thomastrinn opened this issue 11 months ago • 3 comments

Description

When using logback-ecs-encoder with SLF4J's MDC to set array-type fields (like event.type and event.category), the values are serialized as string literals instead of proper JSON arrays. While MDC only supports String values by design, the ECS encoder could detect and properly format string values that represent arrays for fields that are defined as arrays in the ECS specification.

Current Behavior

When setting an array value in MDC (which only accepts strings):

MDC.put("event.type", Arrays.asList("connection", "allowed").toString());

The current output in logs:

{
  "@timestamp": "2025-01-08T13:00:53.318Z",
  "event.type": "[connection, allowed]",
  // other fields...
}

Expected Behaviour

The log output should contain a proper JSON array according to ECS specification:

{
  "@timestamp": "2025-01-08T13:00:53.318Z",
  "event.type": ["connection", "allowed"],
  // other fields...
}

Technical Details

The issue is in EcsJsonSerializer.serializeMDC() where all MDC values are treated as string literals:

builder.append("\":\"");
JsonUtils.quoteAsString(toNullSafeString(String.valueOf(entry.getValue())), builder);
builder.append("\",");

While we understand that MDC only supports string values, the ECS encoder could detect and properly format these string values for fields that are defined as arrays in the ECS specification.

Impact

This limitation affects any field that should be an array according to ECS specification, particularly:

  • event.type
  • event.category
  • tags
  • labels

This makes it difficult to use the library with standard Java collections for fields that should be arrays according to the ECS specification.

Suggested Solution

The serializer could:

  • Check if the field name matches known array fields from ECS specification
  • Check if the string value represents a list (e.g., starts with '[' and ends with ']')
  • Parse and format such values as proper JSON arrays

Example implementation approach:

private static final Set<String> ARRAY_FIELDS = Set.of(
            "event.type",
            "event.category",
            "tags",
            "labels"
);

public static void serializeMDC(StringBuilder builder, Map<String, ?> properties) {
    if (properties != null && !properties.isEmpty()) {
        for (Map.Entry<String, ?> entry : properties.entrySet()) {
            builder.append('\"');
            String key = entry.getKey();
            JsonUtils.quoteAsString(key, builder);

            String value = toNullSafeString(String.valueOf(entry.getValue()));
            if (value.startsWith("[") && value.endsWith("]")) {
                List<String> items = Arrays.stream(
                                value.substring(1, value.length() - 1)
                                        .split(","))
                        .map(String::trim)
                        .collect(Collectors.toList());

                builder.append("\":");
                builder.append(formatAsJsonArray(items));
                builder.append(",");
            } else {
                builder.append("\":\"");
                JsonUtils.quoteAsString(toNullSafeString(String.valueOf(entry.getValue())), builder);
                builder.append("\",");
            }
        }
    }
}

Environment

  • logback-ecs-encoder version: 1.6.0
  • slf4j-api version: 2.0.9
  • Java version: 11
  • Logback version: 1.4.12

thomastrinn avatar Jan 08 '25 14:01 thomastrinn