Log-File input fails with content-splitter = PATTERN

[trobers]

Hi,

with the following collector config

  graylog-server-log {
    type = "file"
    path = "/var/log/graylog-server/server.log"
    content-splitter = "PATTERN"
    content-splitter-pattern = "^\\d{4}-\\d{2}-\\d{2}T"
  }

the collector transmitted only the line 1 to 3. The log-line 4 would never send, because the pattern does not match.

2015-06-22T15:05:31.715+02:00 INFO  [Log] Rolled new log segment for 'messagejournal-0' in 1 ms.
2015-06-22T15:06:11.851+02:00 INFO  [Log] Scheduling log segment 2470544171 for log messagejournal-0 for deletion.
2015-06-22T15:07:11.851+02:00 INFO  [Log] Deleting segment 2470544171 from log messagejournal-0.
2015-06-22T15:07:11.894+02:00 INFO  [OffsetIndex] Deleting index /var/lib/graylog-server/journal/messagejournal-0/00000000002470544171.index.deleted

These log-line will only send if another log-line will be written to the logfile.

In this case for example it could be happen, that a panic message from an application never transmit to the graylog server.

Is there a solution for this case like a configuration item or something?

[bernd]

This is currently a limitation of the pattern splitter. We will investigate if there is a good solution for this.

Thank you for the report!

[jamiegeyer]

This is a critical roadblock for me in using the collector. It would be nice to see an option to include or exclude stack traces automatically. As it stands right now I can't do either effectively - if I use a newline I get every line of the trace thrown in as a separate entry. If I use a pattern splitter I don't get a message sent until the next message is logged - which in some cases for us could be days.

[bernd]

The only workaround I currently see is to have some kind of timeout after which the buffer is flushed. I will look into this.

[jamiegeyer]

I was thinking exactly the same thing. This would solve a bunch of problems for us.

Thanks!

[dwightkjones2]

@jamiegeyer How did you implement the timeout?

[NathanChristie]

Are there any updates on this issue; it may end up being a go or no-go for graylog. If the last log message is a critical one, we will never see it.

[BMacster]

Any updates on this? This can really allow missing something if logging only fires every few minutes, or worse, hours could pass before a message is fired to graylog.

[bernd]

@NathanChristie We will fix this, yes. I cannot give you a date unfortunately. Regarding the Graylog go, no-go decision, you don't need the Collector to run Graylog. There are alternative log shippers that can send local log files to Graylog.

@BMacster As said, we will fix this eventually. Just can't give a date right now.

[keitopso]

I would also be very pleased if this bug would corrected shortly. My suggestion would be to use a configurable timer.

For example: pattern-timer = 5

If 5 seconds last "incomplete" Log event is nothing more happened in the log file transfer it.

[ygorth]

Finally I've found the reason of my problems. I will wait for this fix. Thanks @bernd!

[Spitfire1900]

I'm going to give a +1 to @bernd's idea of having a buffer flush. If data exists in buffer between polling interval 1 and 2 and the content-splitter pattern does not appear in said buffer between those intervals flush the buffer.

[gn-ley]

This feature should not be recommended without a big warn sign in the docs. Some people may rely on it.

[ghost]

Agree. I have spent way too much time troubleshooting my pattern before I figured out this issue.

[mohanrao]

@bernd can we expect a fix this?

[joschi]

@mohanrao The Graylog Collector has been deprecated. I recommend using nxlog or filebeat to ingest text files and them to Graylog.

[mohanrao]

@joschi Both of them don't have support for AIX

[joschi]

@mohanrao Logstash (using JRuby and thus running on AIX) could be another viable replacement in this case.

[mohanrao]

@joschi Does Logstash has SSL ability? My another requirement is GELF TCP with SSL

[joschi]

@mohanrao Please move this discussion to our public mailing list or join the #graylog channel on freenode IRC.

Thank you!