Configuration for formatting Throwable stack traces

[HaloFour]

I'd like to see more options for how stack traces are formatted in the ECS logs to help in reducing the size of logs.

Some examples:

1. Maximum depth
2. Include/exclude stack frames based on a pattern
3. Shorten class names by abbreviating package names to fit within a specified length
4. Root cause first or last

Ideally a configuration option could override the formatting of the stack trace through an interface implementation provided by the application. That implementation could return a String[] or Stream<String> of the formatted stack frames which are then added to the ECS log either as a JSON string or an array of strings.

I'd be happy to work on a PR for this if there is interest in it being included.

[michaelmcfadyensky]

I have the same requirement as this and would preferably use the ecs encoder with our logback.xml, if possible. I'm currently using the logback logstash encoder along with a ShortenedThrowableConverter to achieve this requirement at the moment.

Would it be possible to expose a property in the ecs-encoder to allow the user to configure any ThrowableConverter in their logback file? For example:

<encoder class="co.elastic.logging.logback.EcsEncoder">
    <throwableConverter class="net.logstash.logback.stacktrace.ShortenedThrowableConverter">
        <exclude>some.package.to.exclude.*</exclude>
        <maxLength>13400</maxLength>
    </throwableConverter>
</encoder>

[felixbarny]

That sounds like a useful feature!

The easiest way to integrate that would be to filter the output of Throwable#printStackTrace on a line-by-line basis, using a custom PrintWriter, similar to this

https://github.com/elastic/ecs-logging-java/blob/80e1f442c8cd77c031cfec52942d879e1b628a7f/ecs-logging-core/src/main/java/co/elastic/logging/EcsJsonSerializer.java#L235-L251

Some suggestions for configuration options:

• excludeContaining: using StringBuilder.indexOf(String)
• excludePattern: slower, but more powerful, using regix
• maxLines: the max number of lines of the output. Does not chop off a line as maxLength would do.

[michaelmcfadyensky]

I found the easiest way to integrate this is to reuse the existing ThrowableHandlingConverter interface that comes with logback. This allows us to plug in any existing implementation of this interface with out having the re-implement it for this specific encoder. Though the downside is that this solution is specific to the logback encoder.

This commit solved the issue I highlighted in my comment above for the logback encoder. I know this may not be the solution you guys would want to go with but it would be good to get feedback on it.

[felixbarny]

While this is indeed only a solution that works for logback and no generic one, I think there's great value in being able to re-use existing ThrowableHandlingConverters. This also doesn't restrict us from adding a more generic/logger implementation agnostic way of filtering stack frames in the future.

I'd happily accept a PR adding support for custom ThrowableHandlingConverters.

[rdifrango]

I also would like to see this feature and I wonder if there's a way to tap into the existing capabilities of logback and log4j2 as they already support syntax that is roughly:

ex{depth}
exception{depth}
throwable{depth}

ex{depth, evaluator-1, ..., evaluator-n}
exception{depth, evaluator-1, ..., evaluator-n}
throwable{depth, evaluator-1, ..., evaluator-n}

or

xEx{depth}
xException{depth}
xThrowable{depth}

xEx{depth, evaluator-1, ..., evaluator-n}
xException{depth, evaluator-1, ..., evaluator-n}
xThrowable{depth, evaluator-1, ..., evaluator-n}

See:

https://logback.qos.ch/manual/layouts.html https://logging.apache.org/log4j/2.x/manual/layouts.html

[HaloFour]

@rdifrango

I also would like to see this feature and I wonder if there's a way to tap into the existing capabilities of logback and log4j2 as they already support syntax that is roughly:

I've started this PR to allow specifying a pattern to be applied when rendering error.stack_trace that allows you to use one of the built-in converters or to use your own: https://github.com/elastic/ecs-logging-java/pull/177