Local class incompatible issue with logstash-1.4.2

[eugenetaranov]

Hi Jurriaan, I am having an issue with log42 input, please see below

Accepted connection {:client=>"172.31.0.251:45698", :server=>"0.0.0.0:1530", :level=>:debug, :file=>"logstash/inputs/log4j2.rb", :line=>"151"} Closing connection {:client=>"172.31.0.251:45698", :exception=>#<IOError: org.apache.logging.log4j.Level; local class incompatible: stream classdesc serialVersionUID = 3077535362528045615, local class serialVersionUID = 1581082>, :level=>:debug, :file=>"logstash/inputs/log4j2.rb", :line=>"116"}

Logstash version logstash-1.4.2-1_2c0f5a1.noarch, log4j2 release-5.0.0/logstash-input-log4j2-5.0.zip

I have tried it also with logstash-1.5.0.rc2-1.noarch but installation failed: /opt/logstash/bin/plugin install logstash-input-log4j2 Validating logstash-input-log4j2 Installing logstash-input-log4j2 Plugin version conflict, aborting Installation aborted

/opt/logstash/bin/plugin list |grep log4j logstash-input-log4j

Can you please look ? Thanks

UPD: It may be important, I have installed it copying jars from vendor/* to /opt/logstash/vendor and log4j2.rb into /opt/logstash/lib/logstash/inputs

[jurmous]

Hi Eugene,

For me the 1.4.2 version is working and the debug logs are unfortunately not that descriptive to point out the issue. Did you also try to keep the jars in their normal folder? What is the Java log4j2 side reporting? And what setup for log4j2 did you use?

The plugin was not yet compatible with the latest 1.5.0 rc release. (I developed it against the beta and they changed the api a bit) I just updated the plugin to support RC2 so you can try again to install it.

[eugenetaranov]

Hi Jurriaan,

I have tried again following your instructions, just unzipped version for logstash 1.4 and run as below:

/opt/logstash/bin/logstash --debug --pluginpath /root/logstash-input-log4j2 -f /etc/logstash/conf.d/logstash.conf

however it failed with error below:

Could not find any plugins in "/root/logstash-input-log4j2"
I tried to find files matching the following, but found none:
  /root/logstash-input-log4j2/logstash/{inputs,codecs,filters,outputs}/*.rb {:level=>:warn}

Then I moved log4j2 input up like that:

.
├── logstash
│   └── inputs
│       └── log4j2.rb
├── README.md
└── vendor
    ├── disruptor-3.3.0.jar
    ├── log4j-api-2.1.jar
    └── log4j-core-2.1.jar

and started again. A minute later it started accepting client connections but they failed shortly:

org.apache.logging.log4j.Level; local class incompatible: stream classdesc serialVersionUID = 3077535362528045615, local class serialVersionUID = 1581082 {:level=>:debug, :exception=>IOError, :backtrace=>["org/jruby/ext/jruby/JRubyObjectInputStream.java:58:in `read_object'", "/root/logstash-input-log4j2/logstash/inputs/log4j2.rb:85:in `handle_socket'", "org/jruby/RubyKernel.java:1521:in `loop'", "/root/logstash-input-log4j2/logstash/inputs/log4j2.rb:83:in `handle_socket'", "/root/logstash-input-log4j2/logstash/inputs/log4j2.rb:152:in `run'"], :file=>"logstash/inputs/log4j2.rb", :line=>"115"}
Closing connection {:client=>"172.31.0.30:37835", :exception=>#<IOError: org.apache.logging.log4j.Level; local class incompatible: stream classdesc serialVersionUID = 3077535362528045615, local class serialVersionUID = 1581082>, :level=>:debug, :file=>"logstash/inputs/log4j2.rb", :line=>"116"}

Then I attempted to run with logstash 1.5. Now it installed plugin with no error:

/opt/logstash/bin/plugin install logstash-input-log4j2
Validating logstash-input-log4j2
Installing logstash-input-log4j2
Installation successful

But the issue was the same, it starts failing on any connection arriving to log4j2 input:

Accepted connection {:client=>"172.31.0.30:37891", :server=>"0.0.0.0:1530", :level=>:debug, :file=>"logstash/inputs/log4j2.rb", :line=>"151", :method=>"run"}
Accepted connection {:client=>"172.31.0.29:58833", :server=>"0.0.0.0:1530", :level=>:debug, :file=>"logstash/inputs/log4j2.rb", :line=>"151", :method=>"run"}
org.apache.logging.log4j.Level; local class incompatible: stream classdesc serialVersionUID = 3077535362528045615, local class serialVersionUID = 1581082 {:level=>:debug, :exception=>IOError, :backtrace=>["org/jruby/ext/jruby/JRubyObjectInputStream.java:58:in `read_object'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-log4j2-5.1-java/lib/logstash/inputs/log4j2.rb:85:in `handle_socket'", "org/jruby/RubyKernel.java:1507:in `loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-log4j2-5.1-java/lib/logstash/inputs/log4j2.rb:83:in `handle_socket'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-log4j2-5.1-java/lib/logstash/inputs/log4j2.rb:152:in `run'"], :file=>"logstash/inputs/log4j2.rb", :line=>"115", :method=>"handle_socket"}
Closing connection {:client=>"172.31.0.29:58833", :exception=>#<IOError: org.apache.logging.log4j.Level; local class incompatible: stream classdesc serialVersionUID = 3077535362528045615, local class serialVersionUID = 1581082>, :level=>:debug, :file=>"logstash/inputs/log4j2.rb", :line=>"116", :method=>"handle_socket"}

Here are some config files attached. logstash conf omitting filters section:

input {
  log4j2 {
    mode => "server"
    port => 1530
    type => "java"
  }
}
output {
    if [type] == "messages" {
        elasticsearch {
            index => "syslog-%{+YYYY.MM.dd}"
            host => "127.0.0.1"
            port => 9200
            protocol => http
        }
    } else if [type] == "java" {
        stdout { codec => rubydebug }
    }
}

log4j2.xml:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
  <appenders>
    <Console name="Console" target="SYSTEM_OUT">
       <PatternLayout pattern="%d{HH:mm:ss.SSS} %-5p [%c{1}] %m%n" />
    </Console>
    <RollingFile name="File"
                 fileName="${sys:catalina.home}/logs/app.log"
                 filePattern="${sys:catalina.home}/logs/app.%d{yyyy-MM-dd}.log.gz">
      <PatternLayout pattern="%d{HH:mm:ss.SSS} %-5p [%c{1}] %m%n" />
      <TimeBasedTriggeringPolicy />
      <DefaultRolloverStrategy max="5"/>
    </RollingFile>
    <Socket  name="Socket"
             host="logstash-host"
             port="1530">
      <SerializedLayout />
    </Socket>
  </appenders>
  <loggers>
      <logger name="com.company" level="DEBUG" />
      <logger name="com.company.app.importer" level="DEBUG" />
    <root level="INFO">
      <appender-ref ref="Console" />
      <appender-ref ref="File" />
      <appender-ref ref="Socket" />
    </root>
  </loggers>
</configuration>

Thank you.

[jurmous]

I think you run an old version of log4j2. Your log shows it tries to send a org.apache.logging.log4j.Level with serialVersionUID = 3077535362528045615 On google I found this is the serialVersionUID of Level in Log4j2 2.0 RC1 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.logging.log4j/log4j-api/2.0-rc1/org/apache/logging/log4j/Level.java

The exception also shows it cannot match with org.apache.logging.log4j.Level with serialVersionUID = 1581082 in the plugin which seems to be the serialVersionUID of Level in the current Log4j2: https://git-wip-us.apache.org/repos/asf?p=logging-log4j2.git;a=blob;f=log4j-api/src/main/java/org/apache/logging/log4j/Level.java;h=3698682977f3ca3321c96f273d27a675f9c32733;hb=HEAD

So which version of Log4j2 are you running and can you try to update it if it is not the latest?