Support for Traefik log (JSON format)

[just5ky]

GoAccess error log

GOAN v1.1.8
NGINX SETUP...
NGINX BASIC AUTH
-------------------------------
None
LOADING TRAEFIK LOGS
-------------------------------
	Adding proxy logs...
 	 Filename: /opt/log/access.log | Read = yes
Found (1) proxy logs...
EXCLUDE IPS
-------------------------------
None
DEBUG
-------------------------------
ON
Setting GeoIP Database
-------------------------------
DEFAULT
SKIP ARCHIVED LOGS
-------------------------------
FEATURE NOT AVAILABLE FOR TRAEFIK
RUN TRAEFIK GOACCESS
Cleaning up resources...
==24== GoAccess - version 1.6.3 - Sep 28 2022 02:28:16
==24== Config file: /goaccess-config/goaccess.conf
==24== https://goaccess.io - <[email protected]>
==24== Released under the MIT License.
==24==
==24== FILE: /opt/log/access.log
==24== Parsed 10 lines producing the following errors:
==24==
==24== Token for '%h' specifier is NULL.
==24== Token for '%h' specifier is NULL.
==24== Token for '%h' specifier is NULL.
==24== Token for '%h' specifier is NULL.
==24== Token for '%h' specifier is NULL.
==24== Token for '%h' specifier is NULL.
==24== Token for '%h' specifier is NULL.
==24== Token for '%h' specifier is NULL.
==24== Token for '%h' specifier is NULL.
==24== Token for '%h' specifier is NULL.
==24==
==24== Format Errors - Verify your log/date/time format

docker compose

  goaccess:
    image: xavierh/goaccess-for-nginxproxymanager:latest
    container_name: goaccess
    restart: unless-stopped
    networks:
      - proxy
    ports:
        - '7880:7880'
    environment:
        - PUID=$PUID
        - PGID=$PGID
        - TZ=$TZ       
        #- SKIP_ARCHIVED_LOGS=False #optional
        - DEBUG=True #optional
        #- BASIC_AUTH=False #optional
        #- BASIC_AUTH_USERNAME=user #optional
        #- BASIC_AUTH_PASSWORD=pass #optional   
        #- EXCLUDE_IPS=127.0.0.1 #optional
        - LOG_TYPE=TRAEFIK #optional
    volumes:
        - $DOCKERDIR/traefik:/opt/log:ro
    labels:
      homepage.group: Admin
      homepage.name: Goaccess
      homepage.icon: goaccess.png
      homepage.href: https://access.$DOMAINNAME
      traefik.http.routers.goaccess.rule: Host(`access.$DOMAINNAME`)
      traefik.http.routers.goaccess.entryPoints: https  
      traefik.enable: true

Traefik JSON log sample

{"ClientAddr":"10.244.67.0:29148","ClientHost":"103.246.X.X","ClientPort":"29148","ClientUsername":"-","DownstreamContentSize":2485,"DownstreamStatus":200,"Duration":12662676,"OriginContentSize":2485,"OriginDuration":12492912,"OriginStatus":200,"Overhead":169764,"RequestAddr":"app.justsky.TLD","RequestContentSize":0,"RequestCount":32,"RequestHost":"app.justsky.TLD","RequestMethod":"GET","RequestPath":"/","RequestPort":"-","RequestProtocol":"HTTP/2.0","RequestScheme":"https","RetryAttempts":0,"RouterName":"https-homepage@docker","ServiceAddr":"172.19.0.5:3000","ServiceName":"homepage@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"172.19.0.5:3000","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2022-10-05T05:09:58.981528734+05:30","StartUTC":"2022-10-04T23:39:58.981528734Z","TLSCipher":"TLS_AES_128_GCM_SHA256","TLSVersion":"1.3","entryPointName":"https","level":"info","msg":"","time":"2022-10-05T05:09:58+05:30"}
{"ClientAddr":"10.244.67.0:28982","ClientHost":"103.246.X.X","ClientPort":"28982","ClientUsername":"-","DownstreamContentSize":0,"DownstreamStatus":304,"Duration":9141345,"OriginContentSize":0,"OriginDuration":9093492,"OriginStatus":304,"Overhead":47853,"RequestAddr":"app.justsky.TLD","RequestContentSize":0,"RequestCount":33,"RequestHost":"app.justsky.TLD","RequestMethod":"GET","RequestPath":"/api/validate","RequestPort":"-","RequestProtocol":"HTTP/2.0","RequestScheme":"https","RetryAttempts":0,"RouterName":"https-homepage@docker","ServiceAddr":"172.19.0.5:3000","ServiceName":"homepage@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"172.19.0.5:3000","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2022-10-05T05:09:59.6162877+05:30","StartUTC":"2022-10-04T23:39:59.6162877Z","TLSCipher":"TLS_AES_128_GCM_SHA256","TLSVersion":"1.3","entryPointName":"https","level":"info","msg":"","time":"2022-10-05T05:09:59+05:30"}
{"ClientAddr":"10.244.67.0:29112","ClientHost":"103.246.X.X","ClientPort":"29112","ClientUsername":"-","DownstreamContentSize":0,"DownstreamStatus":304,"Duration":15817575,"OriginContentSize":0,"OriginDuration":15757734,"OriginStatus":304,"Overhead":59841,"RequestAddr":"app.justsky.TLD","RequestContentSize":0,"RequestCount":37,"RequestHost":"app.justsky.TLD","RequestMethod":"GET","RequestPath":"/api/hash","RequestPort":"-","RequestProtocol":"HTTP/2.0","RequestScheme":"https","RetryAttempts":0,"RouterName":"https-homepage@docker","ServiceAddr":"172.19.0.5:3000","ServiceName":"homepage@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"172.19.0.5:3000","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2022-10-05T05:09:59.619260536+05:30","StartUTC":"2022-10-04T23:39:59.619260536Z","TLSCipher":"TLS_AES_128_GCM_SHA256","TLSVersion":"1.3","entryPointName":"https","level":"info","msg":"","time":"2022-10-05T05:09:59+05:30"}
{"ClientAddr":"10.244.67.0:28896","ClientHost":"103.246.X.X","ClientPort":"28896","ClientUsername":"-","DownstreamContentSize":0,"DownstreamStatus":304,"Duration":20796032,"OriginContentSize":0,"OriginDuration":20748944,"OriginStatus":304,"Overhead":47088,"RequestAddr":"app.justsky.TLD","RequestContentSize":0,"RequestCount":34,"RequestHost":"app.justsky.TLD","RequestMethod":"GET","RequestPath":"/api/widgets","RequestPort":"-","RequestProtocol":"HTTP/2.0","RequestScheme":"https","RetryAttempts":0,"RouterName":"https-homepage@docker","ServiceAddr":"172.19.0.5:3000","ServiceName":"homepage@docker","ServiceURL":{"Scheme":"http","Opaque":"","User":null,"Host":"172.19.0.5:3000","Path":"","RawPath":"","OmitHost":false,"ForceQuery":false,"RawQuery":"","Fragment":"","RawFragment":""},"StartLocal":"2022-10-05T05:09:59.616384168+05:30","StartUTC":"2022-10-04T23:39:59.616384168Z","TLSCipher":"TLS_AES_128_GCM_SHA256","TLSVersion":"1.3","entryPointName":"https","level":"info","msg":"","time":"2022-10-05T05:09:59+05:30"}
{"ClientAddr":"10.244.67.0:29162","ClientHost":"103.246.X.X","ClientPort":"29162","ClientUsername":"-","DownstreamContentSize":0,"DownstreamStatus":304,"Duration":22514326,"OriginContentSize":0,"OriginDuration":22451930,"OriginStatus":304,"Overhead":62396,"RequestAddr":"app.justsky.TLD","RequestContentSize":0,"RequestCount":36,"RequestHost":"app.justsky.TLD","RequestMethod":"GET","RequestPath":"/api/bookmarks","RequestPort":"-","RequestProtocol":"HTTP/2.0","RequestScheme":"https","RetryAttempts":0,"RouterName":"https-homepage@docker","ServiceAddr":"172.19.0.5:3000","ServiceName":"homepage@docker","ServiceURL":

[xavier-hernandez]

@Just5KY I just looked at the documentation for TRAEFIK because it looks like for some reason I lost my sample files from my installation a way back. It looks like there is an option for JSON which I can only assume you have on if so I don't think that is going to work and I don't think a log format can be created for that.

filePath: "/path/to/access.log" format: json

https://doc.traefik.io/traefik/observability/access-logs/

The container is looking for something like this.

<remote_IP_address> - <client_user_name_if_available> [] "<request_method> <request_path> <request_protocol>" <origin_server_HTTP_status> <origin_server_content_size> "<request_referrer>" "<request_user_agent>" <number_of_requests_received_since_Traefik_started> "<Traefik_router_name>" "<Traefik_server_URL>" <request_duration_in_ms>ms

[just5ky]

So I should change the log schema to default other than JSON?

[xavier-hernandez]

My understanding from the support page and my experience is that just remove "format: json" and it should start logging in a common format.

[just5ky]

Yes, I understand that, but JSON logging is easier for ingesting logs in ELK and Graylog. That's why I wanted JSON support.

[xavier-hernandez]

Can you give me a couple of days to look into this? Are you able to provide me a larger log file?

[just5ky]

Sure, let me hide IP and tokens. I will send it to you.

[xavier-hernandez]

I grabbed it, and deleted your comment with the file just FYI.

[xavier-hernandez]

@Just5KY sorry had some issues recently. I'm back to updating the repo. Do you still need this?

If so, I don't have your file anymore. Can you provide it if you can?

Thanks.

[just5ky]

Yup, i can re-upload the file for you

[xavier-hernandez]

@Just5KY did you still want this? I didn't get an update. Thanks!

[xavier-hernandez]

Closing this for now... open again if you want me to work on it.