geoblock icon indicating copy to clipboard operation
geoblock copied to clipboard

Traefik Logs Parser Fails on Non-JSON Logs

Open qymab opened this issue 10 months ago • 4 comments
trafficstars

The crowdsecurity/traefik-logs parser fails when encountering non-JSON log lines, such as those generated by GeoBlock. These logs are plain text and not in JSON format, causing the UnmarshalJSON function to throw an error. This results in the following error message:

level=warning msg="failed to run filter : invalid character 'I' looking for beginning of value (1:1)
| UnmarshalJSON(evt.Parsed.message, evt.Unmarshaled, \"traefik\") in [\"\", nil]
| ^" id=blue-grass name=child-crowdsecurity/traefik-logs stage=s01-parse

This happens because the parser attempts to process all log lines, including non-JSON lines, which are incompatible with the UnmarshalJSON function.


Steps to Reproduce:

  1. Configure Traefik to log both access logs and GeoBlock logs in the same file.
  2. Enable the crowdsecurity/traefik-logs parser in CrowdSec.
  3. Feed the combined log file to CrowdSec.
  4. Observe the parser failing on non-JSON log lines (e.g., INFO: GeoBlock).

Expected Behavior:

The crowdsecurity/traefik-logs parser should:

  • Ignore non-JSON log lines (e.g., GeoBlock logs) by default.
  • Process only valid JSON log lines from Traefik.

Actual Behavior:

The parser fails when encountering non-JSON log lines, causing errors like:

invalid character 'I' looking for beginning of value

Environment:

  • CrowdSec Version: v1.6.4
  • GeoBlock Version: v0.2.8
  • Traefik Version: v3
  • Operating System: Docker/Debian

Additional Context:

GeoBlock logs are plain text logs generated by Traefik, such as:

INFO: GeoBlock: 2024/12/26 11:36:01 allow local IPs: true
INFO: GeoBlock: 2024/12/26 11:36:01 log local requests: false

These logs are not in JSON format and should not be processed by the crowdsecurity/traefik-logs parser. Including a filter to exclude non-JSON lines would resolve this issue and prevent unnecessary errors.


qymab avatar Dec 26 '24 08:12 qymab

I think you got the wrong repo, or do you expect any actions from me?

PascalMinder avatar Dec 26 '24 09:12 PascalMinder

I think you got the wrong repo, or do you expect any actions from me?

The logs from GeoBlock could be made more compatible with the way Traefik's log parser functions in CrowdSec.

qymab avatar Dec 26 '24 10:12 qymab

Can you provide me with an example how the log format should look like?

PascalMinder avatar Jan 19 '25 09:01 PascalMinder

I am unsure about the format that should be used, but since I am utilizing the CrowdSec parser (https://app.crowdsec.net/hub/author/crowdsecurity/log-parsers/traefik-logs), it expects logs to be in either Common Log Format (as defined for Traefik) or JSON format, as outlined in the Traefik documentation: https://doc.traefik.io/traefik/observability/access-logs/#format.

qymab avatar Jan 19 '25 09:01 qymab