Failure to log with SLF4J

[janvanbesien-ngdata]

Describe the bug

SLF4J logging does not work.

Expected Behavior

I expect no SLF4J warnings and working logging.

Current Behavior

SLF4J logging fails to initialize with this output:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

Reproduction Steps

The problem can easily be reproduced by triggering the slf4j initialization in jshell with the AWS SDK bundle jar and an SLF4j impl on the classpath:

$ CLASSPATH=~/.m2/repository/software/amazon/awssdk/bundle/2.26.5/bundle-2.26.5.jar:~/.m2/repository/org/slf4j/slf4j-simple/2.0.7/slf4j-simple-2.0.7.jar jshell
|  Welcome to JShell -- Version 21.0.3
|  For an introduction type: /help intro

jshell> software.amazon.awssdk.thirdparty.org.slf4j.impl.StaticLoggerBinder.getSingleton()
|  Exception java.lang.NoClassDefFoundError: software.amazon.awssdk.thirdparty.org.slf4j.impl.StaticLoggerBinder
|        at StaticLoggerBinder.getSingleton (StaticLoggerBinder.java:72)
|        at (#1:1)

jshell> 

Note that the NoClassDefFoundError is thrown by the StaticLoggerBinder (from the bundle-logging-bridge module) because it tries to load "org.slf4j.impl.StaticLoggerBinder", but that was shaded to "software.amazon.awssdk.thirdparty.org.slf4j.impl.StaticLoggerBinder"? Or maybe the intention is to load the actual org.slf4j.impl.StaticLoggerBinder from slf4j, but that doesn't seem to work either (cfr the slf4j-simpler jar on the classpath).

Possible Solution

not sure.

Additional Information/Context

No response

AWS Java SDK version used

2.26.5

JDK version used

21.0.3

Operating System and version

Fedora 40

[janvanbesien-ngdata]

I just realized that there is no StaticLoggerBinder in slf4j 2.0. So the problem is most likely simply that only slf4j 1.7 is supported? Is an upgrade to slf4j 2.0 on the table?

[debora-ito]

@janvanbesien-ngdata @akshat62

Is there a reason you need the slf4j version inside the SDK to be upgraded?

We have some concerns about 2.x backwards compatibility (https://github.com/aws/aws-sdk-java-v2/pull/5105#issuecomment-2073506625), and upgrading needs a deeper investigation work. Can you upgrade it in your project?

[janvanbesien-ngdata]

Our project already uses slf4j 2.x hence we have the slf4j-api and an slf4j 2.x provider/binding/impl on the classpath. However, this provider is incompatible with the older slf4j-api inside the SDK, hence no logging from the SDK.

My jshell example demonstrates that with the slf4j-simple provider but it's the same problem for other providers.

IIUC, the lack of backwards compatibility between slf4j providers and api that you are concerned about in #5105, is exactly what I am running into. So it is annoying either way: (a) if you upgrade, customers will have to upgrade as well. (b) If you don't upgrade, customers can't upgrade either.

You can't postpone the upgrade for ever, so (a) is the only way forward IMHO.

[debora-ito]

We've added a task to our backlog to investigate if it's possible to upgrade to slf4j 2.x without breaking backwards compatibility.

We don't have a timeline for when the task will be prioritized. In the meantime, anyone who'd like to see this feature supported please add a 👍 to the initial description of the issue.

[bcolyn-ngdata]

I'd like to recommend not shading SLF4J into the bundle jar. SLF4J has an excellent track record of binary compatibility towards clients of slf4j-api (not so much the providers/bindings on the other end, but that is not the role the AWS SDK plays here). Libraries build against slf4j-api 1.7 will run fine if slf4j-api 2.x is on their classpath instead. This way, the SDK would match the environment it is used in.

[luketn]

Would be good to see a clear recommendation here - any update?

Should we use SL4J simple? Any shading changes needed? Let's get the archetype into shape!

[janvanbesien]

What I didn't realize when writing this bug report, is that the AWS bundle uses shading for all it's dependencies. So in theory we could solve our original problem by not relying on the bundle, but on the "lean" jar and just manage the dependencies ourselves (via the bom artefact that you provide).

However, we don't manage this ourselves directly, but it comes in via hadoop, so we'd have to check with the hadoop development team. It seems that they are at least aware of the problem, but don't have an immediate solution either due to conflicts within those dependencies (cfr https://issues.apache.org/jira/browse/SPARK-49508).

Shading slf4j-api and still expecting the logging implementation to be pluggable, is never going to work properly I think.

Not shading slf4j-api is an option, but users would then have to add slf4j-api and an impl of their choice to the classpath themselves. This would be fine for us, but I'm guessing the AWS bundle is also intended to "just work" when used standalone without any classpath modifications.

[glennhj]

I'm trying to build a Spark 4 image that includes hadoop-aws and I think I'm running into this problem. I'm hoping someone here who understands the issue better than me may be able to help.

Spark 4 includes slf4j-api-2.0.16.jar and log4j-slf4j2-impl-2.24.3.jar. The hadoop-aws dependency brings in the AWS Java SDK 2 bundle. The result is misconfigured logging because the binding (log4j-slf4j2-impl-2.24.3.jar) is incompatible with the slf4j 1 implementation included in the AWS bundle. I have a couple of questions:

1. Why does bundle-2.24.6.jar contain org.slf4j and software.amazon.awssdk.thirdparty.org.slf4j? It looks like it contains shaded and unshaded versions of slf4j, and the shaded version includes an implementation. Why is the unshaded version necessary?

2. The AWS docs (https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/logging-slf4j.html#sdk-java-logging-classpath) say to add a dependency on log4j-slf4j2-impl (the bridge between slf4j 2 and Log4j 2) in order to configure the Log4j 2 binding for slf4j. But the bundle is built with slf4j version 1.7.36 (https://github.com/aws/aws-sdk-java-v2/blob/2.31.76/pom.xml#L112), so what use is a binding to slf4j 2?

Any guidance would be greatly appreciated!

[glennhj]

I've looked into this a bit more and have come to the conclusion that it isn't possible to use AWS Java SDK 2 bundle alongside code that may use slf4j 2 API. If true, it would mean Spark 4 + hadoop-aws isn't possible. I'm not at all confident that my reasoning is correct. I'd really appreciate it if someone could check my conclusions.

These are the constraints if you want to deploy AWS Java SDK 2 bundle alongside code that may use slf4j 2 API:

1. You must have a v2 slf4j API on the classpath before the bundle (otherwise you risk runtime errors).
2. You cannot have logging from both the bundle and other code. You must choose one or the other.
3. To get logging from the bundle, add an slf4j v1 provider to the classpath.
4. Or, to get logging from other code, add an slf4j v2 provider to the classpath.

Whilst putting a JAR earlier on the classpath has been effective at influencing the order of class loading in my tests, I don't think there is any dependable technique to influence Class loading order. Hence the conclusion that the AWS Java SDK 2 bundle cannot be deployed with code that use slf4j 2 fluent API.

Below are some examples to illustrate the above.

If you want to deploy the AWS Java SDK 2 bundle alongside code that uses slf4j 2 fluent API you must have v2 API on classpath BEFORE the bundle. Otherwise fluent API logging calls will fail, e.g.

$ CLASSPATH=bundle-2.24.6.jar:slf4j-api-2.0.17.jar jshell
|  Welcome to JShell -- Version 21.0.7
|  For an introduction type: /help intro

jshell> org.slf4j.LoggerFactory.getLogger("test").atInfo().log("test fluent logging")
|  Error:
|  cannot find symbol
|    symbol:   method atInfo()
|  org.slf4j.LoggerFactory.getLogger("test").atInfo().log("test fluent logging")
|  ^----------------------------------------------^

If the slfj 2 API is before the bundle on the classpath then fluent logging doesn't cause an error

$ CLASSPATH=slf4j-api-2.0.17.jar:bundle-2.24.6.jar jshell
|  Welcome to JShell -- Version 21.0.7
|  For an introduction type: /help intro

jshell> org.slf4j.LoggerFactory.getLogger("test").atInfo().log("test fluent logging")
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.

Then you can opt to get logging from the bundle by adding a v1 binding to the classpath. In this case you will get no logging from any other code that uses slf4j.

$ CLASSPATH=slf4j-api-2.0.17.jar:bundle-2.24.6.jar:slf4j-simple-1.7.36.jar jshell
|  Welcome to JShell -- Version 21.0.7
|  For an introduction type: /help intro

jshell> org.slf4j.LoggerFactory.getLogger("test").atInfo().log("test fluent logging")
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
SLF4J(W): Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.
SLF4J(W): Ignoring binding found at [jar:file:/home/glenn.j/repos-core/spark-uw-console/spark-uw/target/dependency/slf4j-simple-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J(W): See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.

jshell> software.amazon.awssdk.thirdparty.org.slf4j.LoggerFactory.getLogger("test").info("test shaded slf4j")
[main] INFO test - test shaded slf4j

jshell> org.slf4j.LoggerFactory.getLogger("test").info("test non-shaded slf4j")

Or you can get logging from other code by adding a v2 binding to the classpath.

(venv) [AAD\glenn.j@a-16nl0s4gllnz5 dependency]$ CLASSPATH=slf4j-api-2.0.17.jar:bundle-2.24.6.jar:slf4j-simple-2.0.17.jar jshell
|  Welcome to JShell -- Version 21.0.7
|  For an introduction type: /help intro

jshell> org.slf4j.LoggerFactory.getLogger("test").atInfo().log("test fluent logging")
[main] INFO test - test fluent logging

jshell> software.amazon.awssdk.thirdparty.org.slf4j.LoggerFactory.getLogger("test").info("test shaded slf4j")
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

jshell> org.slf4j.LoggerFactory.getLogger("test").info("test non-shaded slf4j")
[main] INFO test - test non-shaded slf4j

[janvanbesien]

Your description of the problem is correct and in-line with the original bug report.

In theory (untested), if you have direct control over the dependency to the bundle, you could fix the problem by not depending on software.amazon.awssdk:bundle but only on software.amazon.awssdk:aws-java-sdk-s3 and other things that you might need. That avoids the whole problem introduced by shading.

However, I assume you are depending on the bundle transitively via a dependency on hadoop-aws. In that case, you are in exactly the same situation as me.

I still think it would make sense for the bundle to not include any slf4j impl jars. This is what the documentation link that you found, seems to suggest was always the intention: if you want the bundle to produce logs, add an slf4j impl on the classpath. The only consequence is that standalone usage of the bundle won't produce any logging out of the box anymore.

So IMHO solution:

• upgrade to slf4j 2
• don't include any (shaded or unshaded) slf4j impl in the bundle
• accept that standalone usages won't produce any logging (or explain in the doc how to add an slf4j impl jar for those cases)

[glennhj]

Thanks @janvanbesien . In which case, isn't this a bug rather than a feature request? The bundle will cause runtime errors if deployed alongside code that uses the slf4j 2 fluent API. That feels like a bug to me.

[janvanbesien]

I agree. It was a bug originally. I don't think I have permissions to change labels though.

[glennhj]

In case it's helpful to others, here is what we've done to workaround this bug. We delete the unshaded slf4j 1 API from the AWS sdk bundle during our build.

zip --delete ./target/dependency/bundle-*.jar org/slf4j\*

This avoids the risk of crashes from calls to the slf4j 2 API and enables us to deploy an slf4j 2 API & provider for the benefit of everything other than the bundle. The downside is that the bundle defaults to a NoOp logger, so we get no logging from it. If anyone can think of a better workaround, please let me know!