logstash-codec-protobuf icon indicating copy to clipboard operation
logstash-codec-protobuf copied to clipboard

Error when trying to run with multiple .pb.rb files in logstash-codec-protobuf

Open rv4004 opened this issue 4 years ago • 2 comments

Compiled .proto files with ruby-protoc sample.proto and used the same in logstash config. When using the plugin I get the error below. I have multiple definition files to add but for now i am adding only couple. Any help would be greatly appreciated.

edrs.pb.rb.txt affirmedEdr.pb.rb.txt

edrs.proto.txt affirmedEdr.proto.txt

Error message :

[2020-06-17T08:32:23,744][ERROR][logstash.codecs.protobuf ] Unable to load file: /root/proto2/edrs.pb.rb. Reason: #<RuntimeError: Field already exists for tag: 1> [2020-06-17T08:32:23,759][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (RuntimeError) Field already exists for tag: 1", :backtrace=>["org.logstash.config.ir.CompiledPipeline.(CompiledPipeline.java:119)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:80)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:43)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:82)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$VARARGS(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:82)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:342)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:834)"]} [2020-06-17T08:32:23,786][ERROR][logstash.agent ] An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle Java::JavaLang::IllegalStateException for PipelineAction::Create<main>", :backtrace=>["org/logstash/execution/ConvergeResultExt.java:129:in create'", "org/logstash/execution/ConvergeResultExt.java:57:in add'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:355:in `block in converge_state'"]}

./logstash --version logstash 7.7.1

Logstash config: input { file { path => "/proto/test.proto" start_position => "beginning" codec => protobuf { class_name => "Affirmed" class_file => '/root/proto2/edrs.pb.rb' protobuf_root_directory => "/root/proto2/" protobuf_version => 2 } } }

Please let me know if anything else needed.

rv4004 avatar Jun 17 '20 21:06 rv4004

@rv4004 which version of the codec are you using?

IngaFeick avatar Sep 03 '20 11:09 IngaFeick

logstash-codec-protobuf (1.2.2)

rv4004 avatar Sep 03 '20 12:09 rv4004