syslog-rfc5424-parser icon indicating copy to clipboard operation
syslog-rfc5424-parser copied to clipboard

Issue parsing Heroku Logplex logs

Open guidoiaquinti opened this issue 3 years ago • 1 comments

👋 Hi! I'm probably doing something wrong but it seems this library doesn't parse correctly Heroku Logplex HTTPS drain logs that are defined as:

With HTTPS drains, Logplex buffers log messages and submits batches of them to an HTTPS endpoint via a POST request. The POST body contains Syslog-formatted messages, framed using the Syslog TCP protocol octet counting framing method. These batches are posted with a Content-Type header of application/logplex-1.

ref: https://devcenter.heroku.com/articles/log-drains#https-drains

that should be rfc5424#section-6 + rfc6587#section-3.4.1 compliant.

Taking as example the logs from the link above, even after removing the octect part, it doesn't seems to work:

from syslog_rfc5424_parser import parser
message = "<40>1 2012-11-30T06:45:29+00:00 host app web.3 - State changed from starting to up"
parser.parse(message)

Thank you! 🙇

guidoiaquinti avatar Jan 28 '22 13:01 guidoiaquinti

that lot line looks a lot more like RFC 3164 syslog to me. RFC 5424 requires a "-" if structured data is omitted but I don't see one here. This crate doesn't support parsing RFC 3164 syslog (although I imagine I could add it if there were demand).

You might try syslog_rfc3164, although I've never used it and can't really vouch for it.

Roguelazer avatar Jan 28 '22 19:01 Roguelazer

Hey @guidoiaquinti, was this still something that you needed or were you able to try out syslog_rfc3164?

Justintime50 avatar Mar 14 '23 16:03 Justintime50

Hey @guidoiaquinti, was this still something that you needed or were you able to try out syslog_rfc3164?

👋 It was needed at that time but we've implemented a custom internal solution to work around it.

guidoiaquinti avatar Mar 14 '23 16:03 guidoiaquinti

Sounds good, glad you got something working! I'll close this issue, future travelers can feel free to revive if necessary.

Justintime50 avatar Mar 14 '23 16:03 Justintime50