logstash-filter-geoip icon indicating copy to clipboard operation
logstash-filter-geoip copied to clipboard

[bug] replace mmdb database sometimes results in crash

Open 111andre111 opened this issue 5 years ago • 2 comments

For all general issues, please provide the following details for fast resolution: today there is a need of restarting logstash for reloading the Maxmind geoip databases Referring to this comment: https://github.com/logstash-plugins/logstash-filter-geoip/pull/63#issuecomment-316747366 I would like to raise this now as it would be good to have the possibility to reload the database without restarting logstash, and with that you can update the database and logstash would be quite immediately aware of it.

From implementation perspective there are existing mechanisms already that can be taken as a blueprint from the translate plugin: https://github.com/logstash-plugins/logstash-filter-translate/blob/5cbb57c9a34865856de9ead577c19d42c4b75ade/lib/logstash/filters/dictionary/file.rb#L119 reload_dictionary https://github.com/logstash-plugins/logstash-filter-translate/blob/5cbb57c9a34865856de9ead577c19d42c4b75ade/lib/logstash/filters/dictionary/file.rb#L128 needs_refresh and then offer the 2 configurable settings refresh_interval and if needed refresh_behaviour www.elastic.co/guide/en/logstash/current/plugins-filters-translate.html

111andre111 avatar Nov 23 '20 11:11 111andre111

I think I have to revise this one as I did a short test with two custom geoip databases that had a changing field content and during using copying the mmdb and a running logstash the values changed quite immediately, so logstash is getting aware of it. Sorry for the confusion.

111andre111 avatar Nov 23 '20 13:11 111andre111

When I replace the mmdb under the hood of logstash it seems that it sometimes crashes logstash which should even not happen, so let me reopen this one again.

warning: thread "[main]>worker0" terminated with exception (report_on_exception is true):
java.lang.InternalError: a fault occurred in a recent unsafe memory access operation in compiled Java code
	at com.maxmind.db.Reader.resolveDataPointer(com/maxmind/db/Reader.java:242)
	at com.maxmind.db.Reader.get(com/maxmind/db/Reader.java:150)
	at com.maxmind.geoip2.DatabaseReader.get(com/maxmind/geoip2/DatabaseReader.java:151)
	at com.maxmind.geoip2.DatabaseReader.city(com/maxmind/geoip2/DatabaseReader.java:202)
	at org.logstash.filters.GeoIPFilter.retrieveCityGeoData(org/logstash/filters/GeoIPFilter.java:191)
	at org.logstash.filters.GeoIPFilter.handleEvent(org/logstash/filters/GeoIPFilter.java:144)
	at jdk.internal.reflect.GeneratedMethodAccessor8.invoke(jdk/internal/reflect/GeneratedMethodAccessor8)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:564)
	at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:456)
	at org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:317)
	at Users.andreletterer.Downloads.tools.logstash_minus_7_dot_7_dot_0.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_filter_minus_geoip_minus_6_dot_0_dot_3_minus_java.lib.logstash.filters.geoip.filter(Users/andreletterer/Downloads/tools/logstash_minus_7_dot_7_dot_0/vendor/bundle/jruby/$2_dot_5_dot_0/gems/logstash_minus_filter_minus_geoip_minus_6_dot_0_dot_3_minus_java/lib/logstash/filters//Users/andreletterer/Downloads/tools/logstash-7.7.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-geoip-6.0.3-java/lib/logstash/filters/geoip.rb:109)
	at Users.andreletterer.Downloads.tools.logstash_minus_7_dot_7_dot_0.logstash_minus_core.lib.logstash.filters.base.do_filter(Users/andreletterer/Downloads/tools/logstash_minus_7_dot_7_dot_0/logstash_minus_core/lib/logstash/filters//Users/andreletterer/Downloads/tools/logstash-7.7.0/logstash-core/lib/logstash/filters/base.rb:159)
	at Users.andreletterer.Downloads.tools.logstash_minus_7_dot_7_dot_0.logstash_minus_core.lib.logstash.filters.base.multi_filter(Users/andreletterer/Downloads/tools/logstash_minus_7_dot_7_dot_0/logstash_minus_core/lib/logstash/filters//Users/andreletterer/Downloads/tools/logstash-7.7.0/logstash-core/lib/logstash/filters/base.rb:178)
	at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1809)
	at Users.andreletterer.Downloads.tools.logstash_minus_7_dot_7_dot_0.logstash_minus_core.lib.logstash.filters.base.multi_filter(Users/andreletterer/Downloads/tools/logstash_minus_7_dot_7_dot_0/logstash_minus_core/lib/logstash/filters//Users/andreletterer/Downloads/tools/logstash-7.7.0/logstash-core/lib/logstash/filters/base.rb:175)
	at org.logstash.config.ir.compiler.FilterDelegatorExt.doMultiFilter(org/logstash/config/ir/compiler/FilterDelegatorExt.java:127)
	at org.logstash.config.ir.compiler.AbstractFilterDelegatorExt.multi_filter(org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:134)
	at org.logstash.generated.CompiledDataset1.compute(org/logstash/generated/CompiledDataset1)
	at org.logstash.generated.CompiledDataset2.compute(org/logstash/generated/CompiledDataset2)
	at org.logstash.config.ir.CompiledPipeline$CompiledOrderedExecution.compute(org/logstash/config/ir/CompiledPipeline.java:334)
	at org.logstash.config.ir.CompiledPipeline$CompiledOrderedExecution.compute(org/logstash/config/ir/CompiledPipeline.java:318)
	at org.logstash.execution.WorkerLoop.run(org/logstash/execution/WorkerLoop.java:82)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:62)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:564)
	at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:426)
	at org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:293)
	at Users.andreletterer.Downloads.tools.logstash_minus_7_dot_7_dot_0.logstash_minus_core.lib.logstash.java_pipeline.start_workers(Users/andreletterer/Downloads/tools/logstash_minus_7_dot_7_dot_0/logstash_minus_core/lib/logstash//Users/andreletterer/Downloads/tools/logstash-7.7.0/logstash-core/lib/logstash/java_pipeline.rb:279)
	at org.jruby.RubyProc.call(org/jruby/RubyProc.java:318)
	at java.lang.Thread.run(java/lang/Thread.java:832)

111andre111 avatar Nov 27 '20 12:11 111andre111