logstash-filter-grok

logstash-filter-grok copied to clipboard

Migrated from https://github.com/elastic/logstash/issues/7281 Created by @ror6ax ----- I'm having an issue with grok. I've wrote a string to match `10.155.16.115 - - [30/May/2017:08:53:41 +0000] "GET /sqi HTTP/1.1" 404 16 "-"...

jsvd

Hi, I have been making use of freely available logstash configs and patterns, like this (https://github.com/nxhack/logstash/blob/master/patterns/sshd) awesome one for SSHD. I've copied a small section of the file for illustrative...

nick-george

grok should have a replace functionality like we have for mutate

4

## Usecase I have few custom regex patterns which looks for some sensitive information in the log messages like credit card number, social security number etc. I have applied these...

saurabh8585

relates to https://github.com/logstash-plugins/logstash-filter-grok/issues/82 It'd be great to print a grok filter regex taking a long time to execute. cc @andrewvc

nellicus

Provide a way for end users to define new string mappings in 5.0+

1

With the introduction of text vs. keyword fields in 5.x, it can be a nice feature to provide a way for the end user to define if a field should...

ppf2

 I think we should add a specific Filename Extension for pattern files. - Version: - Operating System: Ubuntu 14.0 - Config File (if you have sensitive info, please remove...

visaxin

Not sure if this is already possible, but it doesn't seem so. Given the String: `Hello:1 Hello:2 Hello:3`, In Perl, you could do: `/Hello:(\d+)/g` , with the Global modifier (`/g`),...

PhaedrusTheGreek

tests fail because LogStash::Environment.pattern_path is not in logstash-core

5

running the specs fail for `logstash-core >= 2.2` ``` /tmp/logstash-filter-grok (git)-[master] % bundle exec rspec Using Accessor#strict_set for specs NoMethodError: undefined method `pattern_path' for LogStash::Environment:Module Grok at /private/tmp/logstash-filter-grok/lib/logstash/filters/grok.rb:226 (root) at...

jsvd

test script: ``` ruby # encoding: utf-8 require 'logstash/event' require 'logstash/environment' require 'spec/filters/grok_spec' grok_base = LogStash::Filters::Grok.new( "match" => ["message", '%{IPORHOST:clientip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response:int} (?:-|%{NUMBER:bytes:int}) %{QS:referrer} %{QS:agent}'])...

jsvd

Why is this filter "slow"? A brief analysis

2

Total time to process the same apache log line 1_000_000 times with `%{COMMONAPACHELOG}`: 93.88s (~10k e/s) The 6 biggest time offenders use 70.46s - 75% of total time: | Action...

jsvd