fluent-logger-java icon indicating copy to clipboard operation
fluent-logger-java copied to clipboard

Published 20 hours ago • verified publisher icon fluent

Null Pointer Exception with slf4j-log4j12-1.7.30 (Fluent-logger incompatible with the new version)

Open anusricorp opened this issue 5 years ago • 3 comments

When running spark-submit with slf4j-log4j12-1.7.18.jar in the ClassPath, FluentLogger works with no issues. When we use version 1.7.30 which comes with Spark3.0, Fluentlogger.getLogger throws a null pointer exception.

This is the offending commit introduced in 1.7.19  which was done to fix a NullPointerException. Its force initializing rootLogger to fix a NPE in multithreaded scenarios looks like. The irony is this is the commit which is causing NullPointerException for us https://github.com/qos-ch/slf4j/commit/111b8e581e247b33bb2fe66bd0118399e0b49a90

Can we upgrade the version of slf4j-api in FluentLogger to latest 1.7.30 and make it compatible?

hdiuser@hn0-d0140a:/usr/hdp/4.1-0/spark2/jars$ spark-submit --version SPARK_MAJOR_VERSION is set to 2, using Spark2 Unexpected problem occured during version sanity check Reported exception: java.lang.NullPointerException at org.slf4j.LoggerFactory.versionSanityCheck(LoggerFactory.java:267) at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:126) at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383) at org.fluentd.logger.sender.RawSocketSender.(RawSocketSender.java:34) at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:72) at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:51) at org.fluentd.logger.FluentLogger.getLogger(FluentLogger.java:40) at com.mdsdclient.MdsLogger.(Unknown Source) at com.log4jappender.common.logger.HdInsightLoggerLinux.(Unknown Source) at com.log4jappender.common.logger.HdInsightLoggerFactory.getLogger(Unknown Source) at com.log4jappender.common.appendercore.AnonymizeLogAppenderCommon.getLogger(Unknown Source) at com.log4jappender.common.appendercore.AnonymizeLogAppenderCommon.(Unknown Source) at com.microsoft.log4jappender.AnonymizeLogAppender.activateOptions(Unknown Source) at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) at org.apache.log4j.LogManager.(LogManager.java:127) at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66) at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72) at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45) at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:222) at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:127) at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111) at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105) at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Exception in thread "main" java.lang.ExceptionInInitializerError at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:72) at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:51) at org.fluentd.logger.FluentLogger.getLogger(FluentLogger.java:40) at com.mdsdclient.MdsLogger.(Unknown Source) at com.log4jappender.common.logger.HdInsightLoggerLinux.(Unknown Source) at com.log4jappender.common.logger.HdInsightLoggerFactory.getLogger(Unknown Source) at com.log4jappender.common.appendercore.AnonymizeLogAppenderCommon.getLogger(Unknown Source) at com.log4jappender.common.appendercore.AnonymizeLogAppenderCommon.(Unknown Source) at com.log4jappender.AnonymizeLogAppender.activateOptions(Unknown Source) at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) at org.apache.log4j.LogManager.(LogManager.java:127) at org.slf4j.impl.Log4jLoggerFactory.(Log4jLoggerFactory.java:66) at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:72) at org.slf4j.impl.StaticLoggerBinder.(StaticLoggerBinder.java:45) at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:222) at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:127) at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111) at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105) at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75) at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83) at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1007) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: java.lang.NullPointerException at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:418) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357) at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383) at org.fluentd.logger.sender.RawSocketSender.(RawSocketSender.java:34) ... 31 more

anusricorp avatar Aug 22 '20 00:08 anusricorp

Thanks for reporting. If you are using Spark, airframe-fluentd can be an alternative for a while, which can be used with Scala https://wvlet.org/airframe/docs/airframe-fluentd

I'll try to update slf4j version too

xerial avatar Aug 25 '20 20:08 xerial

Hi @xerial do we have any update on this issue? We are facing a similar issue with NullPointerException errors in slf4j v1.7.30:

	21/08/05 22:21:01 INFO LineBufferedStream: stdout: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: Failed to instantiate SLF4J LoggerFactory
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: Reported exception:
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: java.lang.NullPointerException
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.reportActualBinding(LoggerFactory.java:349)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.bind(LoggerFactory.java:152)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:417)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.sender.RawSocketSender.<clinit>(RawSocketSender.java:34)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:72)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:51)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:222)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.internal.Logging.initializeLogging(Logging.scala:127)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.internal.Logging.initializeLogIfNecessary(Logging.scala:111)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.internal.Logging.initializeLogIfNecessary$(Logging.scala:105)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit.initializeLogIfNecessary(SparkSubmit.scala:75)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:83)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: Exception in thread "main" java.lang.ExceptionInInitializerError
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:72)
	21/08/05 22:21:01 INFO LineBufferedStream: stdout: 	at org.fluentd.logger.FluentLoggerFactory.getLogger(FluentLoggerFactory.java:51)

Niharikadutta avatar Aug 06 '21 18:08 Niharikadutta

Hi @xerial, is the above NullPointerException is fixed? we are seeing the exact same error.

KeerthiYandaOS avatar Mar 14 '22 15:03 KeerthiYandaOS