Any template ready for GCP?

[ilgrosso]

I was wondering if there is any template available for usage with GCP, according to https://cloud.google.com/logging/docs/agent/configuration#process-payload Thanks!

[vy]

Processing payloads was sort of cryptic to me given my lack of knowledge about its workings. I had a better visual with the StackdriverJsonLayout.java of spring-cloud-gcp-logging project as you pointed in the Log4j mailing list. Let's see which of the fields are resolvable by available LogstashLayout directives and which might necessitate new ones:

if (this.includeMDC) {
	event.getMDCPropertyMap().forEach((key, value) -> {
		if (!FILTERED_MDC_FIELDS.contains(key)) {
			map.put(key, value);
		}
	});
}

mdc directive perfectly suffices here. But in its current form, there is no way to unwrap it at the top level.

if (this.includeTimestamp) {
	map.put(StackdriverTraceConstants.TIMESTAMP_SECONDS_ATTRIBUTE,
			TimeUnit.MILLISECONDS.toSeconds(event.getTimeStamp()));
	map.put(StackdriverTraceConstants.TIMESTAMP_NANOS_ATTRIBUTE,
			TimeUnit.MILLISECONDS.toNanos(event.getTimeStamp() % 1_000));
}

Here I'd rather go with "timestamp": "${json:timestamp}", which is allowed by the GCP documentation. The only catch here is to make sure that timeZoneId is set to UTC.

add(StackdriverTraceConstants.SEVERITY_ATTRIBUTE, this.includeLevel,
		String.valueOf(event.getLevel()), map);

"severity": "${json:level}" (or level:severity?)

add(JsonLayout.THREAD_ATTR_NAME, this.includeThreadName, event.getThreadName(), map);

"thread": "${json:thread:name}"

add(JsonLayout.LOGGER_ATTR_NAME, this.includeLoggerName, event.getLoggerName(), map);

"logger": "${json:logger:name}"

if (this.includeFormattedMessage) {
	String message = event.getFormattedMessage();
	if (this.includeExceptionInMessage) {
		IThrowableProxy throwableProxy = event.getThrowableProxy();
		if (throwableProxy != null) {
			String stackTrace = getThrowableProxyConverter().convert(event);
			if (stackTrace != null && !stackTrace.equals("")) {
				message += "\n" + stackTrace;
			}
		}
	}
	map.put(JsonLayout.FORMATTED_MESSAGE_ATTR_NAME, message);
}

This is interesting. When includeFormattedMessage is true, message field is set to formattedMessage + '\n' + stackTrace, if there is an exception; otherwise, formattedMessage itself. I guess we don't need this. Continue reading...

add(JsonLayout.MESSAGE_ATTR_NAME, this.includeMessage, event.getMessage(), map);

"message": "${json:message}"

add(JsonLayout.CONTEXT_ATTR_NAME, this.includeContextName, event.getLoggerContextVO().getName(), map);

We don't have an equivalent for this in LogstashLayout. It feels like we can just ignore this.

addThrowableInfo(JsonLayout.EXCEPTION_ATTR_NAME, this.includeException, event, map);

"exception": "${json:exception:stackTrace:text}"

addTraceId(event, map);

which translates to

protected String formatTraceId(final String traceId) {
	// Trace IDs are either 64-bit or 128-bit, which is 16-digit hex, or 32-digit hex.
	// If traceId is 64-bit (16-digit hex), then we need to prepend 0's to make a 32-digit hex.
	if (traceId != null && traceId.length() == 16) {
		return "0000000000000000" + traceId;
	}
	return traceId;
}

private void addTraceId(ILoggingEvent event, Map<String, Object> map) {
	if (!this.includeTraceId) {
		return;
	}

	String traceId =
			event.getMDCPropertyMap().get(StackdriverTraceConstants.MDC_FIELD_TRACE_ID);
	if (traceId == null) {
		traceId = TraceIdLoggingEnhancer.getCurrentTraceId();
	}
	if (!StringUtils.isEmpty(traceId)
			&& !StringUtils.isEmpty(this.projectId)
			&& !this.projectId.endsWith("_IS_UNDEFINED")) {
		traceId = StackdriverTraceConstants.composeFullTraceName(
				this.projectId, formatTraceId(traceId));
	}

	add(StackdriverTraceConstants.TRACE_ID_ATTRIBUTE, this.includeTraceId, traceId, map);
}

That's quite some business logic going here. An innocent attempt can be "logging.googleapis.com/trace": "${json:mdc:X-B3-TraceId}", but I doubt if this can cover corner cases (e.g., 32-digit hex issue) mentioned in the code above.

add(StackdriverTraceConstants.SPAN_ID_ATTRIBUTE, this.includeSpanId,
		event.getMDCPropertyMap().get(StackdriverTraceConstants.MDC_FIELD_SPAN_ID), map);

"logging.googleapis.com/spanId": "${json:mdc:X-B3-SpanId}"

if (this.serviceContext != null) {
	map.put(StackdriverTraceConstants.SERVICE_CONTEXT_ATTRIBUTE, this.serviceContext);
}

I don't have a solution here except dragging serviceContext via an MDC field.

if (this.customJson != null && !this.customJson.isEmpty()) {
	for (Map.Entry<String, Object> entry : this.customJson.entrySet()) {
		map.putIfAbsent(entry.getKey(), entry.getValue());
	}
}

This can be achieved by either modifying the template or providing eventTemplateAdditionalFields.

In the light of these findings, at a first glance, the following template might do the trick:

{
  "mdc1": "${json:mdc:key1}",
  "mdc2": "${json:mdc:key2}",
  "mdc3": "${json:mdc:key3}",
  "timestamp": "${json:timestamp}",
  "severity": "${json:level}",
  "thread": "${json:thread:name}",
  "logger": "${json:logger:name}",
  "message": "${json:message}",
  "exception": "${json:exception:stackTrace:text}",
  "logging.googleapis.com/trace": "${json:mdc:X-B3-TraceId}",
  "serviceContext": "${json:mdc:serviceContext}"
}

@ilgrosso, would you mind giving this a try, please? Any other feedback and/or comments are also welcome.

[ilgrosso]

The above looks great, thanks! I'll give it a try early next week and report here.

[ilgrosso]

Thanks for you help @vy , worked like a charm. Important note is also to set prettyPrintEnabled="false".

[vy]

@ilgrosso, before closing the issue, it might be good that we add this as a FAQ to README. Would you mind sharing how did you exactly solve your problem, please? The template JSON, Java code, prepared MDC fields, etc. The more details you provide, the more helpful it will be.

[ilgrosso]

I have no MDC fields (yet?), no Java code to add.

Just added the Maven dependency, grabbed your JSON template as above and defined appenders like as follows:

    <Console name="mainFile" target="SYSTEM_OUT">
      <LogstashLayout dateTimeFormatPattern="yyyy-MM-dd'T'HH:mm:ss.SSSZZZ"
                      timeZoneId="UTC"
                      eventTemplateUri="classpath:logstashGCP.json"
                      prettyPrintEnabled="false"
                      stackTraceEnabled="true"/>
    </Console>

[vy]

How did you deal with logging.googleapis.com/trace, serviceContext, etc. fields? In my JSON template I was expecting them to be injected into MDC.

[ilgrosso]

Ah sorry for not reporting: at the moment I don't need neither logging.googleapis.com/trace nor serviceContext - and they are both reported as null in the actual statements, as expected.

[iamshe]

How can I get custom field say version and it should read the value from other properties file or json file . We don't want to hard code it and wanted to dynamically populate and don't want to put it in mdc.

[vy]

How can I get custom field say version and it should read the value from other properties file or json file . We don't want to hard code it and wanted to dynamically populate and don't want to put it in mdc.

First, I would strongly advise you to switch from LogstashLayout provided by log4j2-logstash-layout artifact to JsonTemplateLayout provided by the official Log4j project in log4j-layout-template-json artifact. LogstashLayout is not developed anymore and JsonTemplateLayout is the successor.

You can define the values either as JVM properties or environment variables. Then you can access these using lookups. This is explained (and demonstrated) in detail in the documentation of both projects.