goaccess
goaccess copied to clipboard
Published
20 hours ago •
allinurl
allinurl
NPM with Geolocation - How is the correct format
trafficstars
Hello, I am new on GoAccess. I need it for control my geoblocking with NPM.
I have installed NPM like here https://www.reddit.com/r/nginxproxymanager/comments/sjf1hw/tutorial_nginx_proxy_manager_together_with_geoip2/
The log is
log_format proxy_geo escape=json '[$time_local] [Client $remote_addr] [$allowed_country $geoip2_data_country_code $geoip2_data_country_name $geoip2_data_region_name $geoip2_data_city_name] "$http_user_agent" '
'$upstream_cache_status $upstream_status $status - $request_method $scheme $host "$request_uri" [Length $body_bytes_sent] [Gzip $gzip_ratio] [Sent-to $server] "$http_referer"';
In NPM I have following logs:
108 -rw-r--r-- 1 root root 105663 Mär 6 08:50 proxy-host-1_access-geo.log
728 -rw-r--r-- 1 root root 741086 Mär 6 08:50 proxy-host-1_access.log
4 -rw-r--r-- 1 root root 630 Mär 6 07:31 proxy-host-1_error.log
12 -rw-r--r-- 1 root root 11002 Mär 6 08:28 proxy-host-2_access.log
8 -rw-r--r-- 1 root root 6169 Mär 6 08:28 proxy-host-2_error.log
0 -rw-r--r-- 1 root root 0 Jan 22 10:16 proxy-host-3_access.log
0 -rw-r--r-- 1 root root 0 Jan 22 10:16 proxy-host-3_error.log
If a add the proxy-host-1_access-geo.log to the config, then GoAccess not starting. How can i show the Geo-Inforamtion in the Visitor Hostnames and IPs and Geo Location Section of GoAccess.
The goaccess.conf is actual following
time-format %H:%M:%S
date-format %d/%b/%Y
log_format [%d:%t %^] - %s %^ - %m %^ %v "%U" [Client %h] [Length %b] [Gzip %^] [Sent-to %^] "%u" "%R"
real-time-html true
log-file /opt/log/proxy-host-1_access.log
log-file /opt/log/proxy-host-2_access.log
log-file /opt/log/proxy-host-3_access.log
[06/Mar/2023:07:19:18 +0000] [Client 149.172.156.150] [yes DE Deutschland Baden-Württemberg Winnenden] "Mozilla/5.0 (Windows) mirall/3.7.3stable-Win64 (build 20230209) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)" 207 207 - PROPFIND https mykey0815.selfhost.eu "/nextcloud/remote.php/dav/files/administrator/" [Length 242] [Gzip ] [Sent-to 192.168.2.40] ""
[06/Mar/2023:07:19:48 +0000] [Client 149.172.156.150] [yes DE Deutschland Baden-Württemberg Winnenden] "Mozilla/5.0 (Windows) mirall/3.7.3stable-Win64 (build 20230209) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)" 207 207 - PROPFIND https mykey0815.selfhost.eu "/nextcloud/remote.php/dav/files/administrator/" [Length 242] [Gzip ] [Sent-to 192.168.2.40] ""
[06/Mar/2023:07:19:48 +0000] [Client 149.172.156.150] [yes DE Deutschland Baden-Württemberg Winnenden] "Mozilla/5.0 (Windows) mirall/3.7.3stable-Win64 (build 20230209) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)" 200 200 - GET https mykey0815.selfhost.eu "/nextcloud/ocs/v2.php/apps/user_status/api/v1/user_status?format=json" [Length 154] [Gzip ] [Sent-to 192.168.2.40] ""
[06/Mar/2023:07:19:48 +0000] [Client 149.172.156.150] [yes DE Deutschland Baden-Württemberg Winnenden] "Mozilla/5.0 (Windows) mirall/3.7.3stable-Win64 (build 20230209) (Nextcloud, windows-10.0.19045 ClientArchitecture: x86_64 OsArchitecture: x86_64)" 304 304 - GET https mykey0815.selfhost.eu "/nextcloud/ocs/v2.php/apps/notifications/api/v2/notifications?format=json" [Length 0] [Gzip ] [Sent-to 192.168.2.40] ""
This should work:
goaccess access.log --log-format='[%x] [%^ %h] [%^ %e] "%u" %^ %s %^ %m %^ %v "%U" [%^ %b] [%^] [%^] [%^] "%R"' --datetime-format='%d/%b/%Y:%H:%M:%S %z' --http-protocol=no
Let me know how it goes.
Just an FYI, upstream fixes the proper detection for those user agents that you posted.
Thank you for your reply. But i have the problem that it not work for me.
Here my docker-compose.yml
version: '3'
services:
goaccess:
image: allinurl/goaccess
container_name: goaccess
restart: always
ports:
- 7890:7890
volumes:
- ./config:/srv/config
- ./html:/srv/report
- ~/npm/data/logs:/srv/logs
command: ["--no-global-config", "--config-file=/srv/config/goaccess.conf", "--log-file=/srv/logs/access.log", -e TZ="Europe/BERLIN", -e "LANG=de_DE.utf8"]
And here the config/goaccess.conf
datetime-format %d/%b/%Y:%H:%M:%S %z
log-format [%x] [%^ %h] [%^ %e] "%u" %^ %s %^ %m %^ %v "%U" [%^ %b] [%^] [%^] [%^] "%R"
real-time-html true
log-file /srv/logs/proxy-host-1_access-geo.log
But the error in the docker-container says:
2023-03-08T05:27:06.058908940Z [SETTING UP STORAGE /srv/logs/access.log] {0} @ {0/s}
2023-03-08T05:27:06.058955328Z ==1== GoAccess - version 1.7.1 - Mar 1 2023 01:28:02
2023-03-08T05:27:06.058959716Z ==1== Config file: /srv/config/goaccess.conf
2023-03-08T05:27:06.058962672Z ==1== https://goaccess.io - <[email protected]>
2023-03-08T05:27:06.058965668Z ==1== Released under the MIT License.
2023-03-08T05:27:06.058968293Z ==1==
2023-03-08T05:27:06.058970678Z ==1== FILE: /srv/logs/proxy-host-1_access-geo.log
2023-03-08T05:27:06.058973132Z ==1== Parsed 10 lines producing the following errors:
2023-03-08T05:27:06.058975928Z ==1==
2023-03-08T05:27:06.058978212Z ==1== Token '06/Mar/2023:07:19:18 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.058989303Z ==1== Token '06/Mar/2023:07:19:48 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.058992148Z ==1== Token '06/Mar/2023:07:19:48 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.058994633Z ==1== Token '06/Mar/2023:07:19:48 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.058997109Z ==1== Token '06/Mar/2023:07:20:18 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.058999623Z ==1== Token '06/Mar/2023:07:20:48 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.059002128Z ==1== Token '06/Mar/2023:07:20:48 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.059004592Z ==1== Token '06/Mar/2023:07:20:48 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.059007047Z ==1== Token '06/Mar/2023:07:21:18 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.059009512Z ==1== Token '06/Mar/2023:07:21:48 +0000' doesn't match specifier '%x'
2023-03-08T05:27:06.059011977Z ==1==
2023-03-08T05:27:06.059014321Z ==1== Format Errors - Verify your log/date/time format
2023-03-08T05:27:06.058922866Z Cleaning up resources...
You can try changing the time/date format as your machine may not support %z. e.g.,
date-format %d/%b/%Y
time-format %T
log-format [%d:%t %^] [%^ %h] [%^ %e] "%u" %^ %s %^ %m %^ %v "%U" [%^ %b] [%^] [%^] [%^] "%R"
I think that is the rigth way - now only missing the html-File which i will open with "http://xxxx.xxx.xxx.xx:7890".
The log is now different:
2023-03-08T16:07:06.368450849Z [SETTING UP STORAGE /srv/logs/access.log] {0} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {3848} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {7850} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {12042} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {15936} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {19918} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {23845} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {27652} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {31358} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {34952} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {38420} @ {38420/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {41965} @ {41965/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {45313} @ {45313/s}
2023-03-08T16:07:06.467953221Z <!DOCTYPE html><html lang='en'><head><meta charset='UTF-8'><meta name='referrer' content='no-referrer'><meta http-equiv='X-UA-Compatible' content='IE=edge'><meta name='google' content='notranslate'><meta name='viewport' content='width=device-width, initial-scale=1'><meta name='robots' content='noindex, nofollow'><link rel='icon' href='data:image/x-icon;base64,AAABAAEAEBAQAAEABAAoAQAAFgAAACgAAAAQAAAAIAAAAAEABAAAAAAAgAAAAAAAAAAAAAAAEAAAAAAAAADGxsYAWFhYABwcHABfAP8A/9dfAADXrwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIiIiIiIiIiIjMlUkQgAiIiIiIiIiIiIiIzJVJEIAAAIiIiIiIiIiIiMyVSRCAAIiIiIiIiIiIiIRERERERERERERERERERERIiIiIiIiIiIgACVVUiIiIiIiIiIiIiIiIAAlVVIiIiIiIiIiIiIiIhEREREREREREREREREREREAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA' type='image/x-icon' /><title>My Awesome Web Stats</title><style>@font-face {font-family: 'fa';src: url(data:application/font-woff;charset=utf-8;base64,d09GRgABAAAAAC2sAAsAAAAALWAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAABPUy8yAAABCAAAAGAAAABgDxIPHGNtYXAAAAFoAAABbAAAAWzzYPN8Z2FzcAAAAtQAAAAIAAAACAAAABBnbHlmAAAC3AAAJ7QAACe0PqRPf2hlYWQAACqQAAAANgAAADYSBhrHaGhlYQAAKsgAAAAkAAAAJAhUBIZobXR4AAAq7AAAAMwAAADMpCoCC2xvY2EAACu4AAAAaAAAAGjyrvuebWF4cAAALCAAAAAgAAAAIAA+AVduYW1lAAAsQAAAAUoAAAFKIhWTsnBvc3QAAC2MAAAAIAAAACAAAwAAAAMDVgGQAAUAAAKZAswAAACPApkCzAAAAesAMwEJAAAAAAAAAAAAAAAAAAAAARAAAAAAAAAAAAAAAAAAAAAAQAAA8pwDwP/AAEADwABAAAAAAQAAAAAAAAAAAAAAIAAAAAAAAwAAAAMAAAAcAAEAAwAAABwAAwABAAAAHAAEAVAAAABQAEAABQAQAAEAIPAH8AnwDfAT8BfwIvA68EHwRvBU8HHwgPCF8I7wlvCi8MDwyfDO8Nrw3PDk8QHxBfEI8QzxEfEh8VzxoPHO8ffx/vJo8pLynP/9//8AAAAAACDwB/AJ8AvwE/AW8CHwOvBB8EbwU/Bx8IDwhfCO8JbwovDA8MnwzvDX8Nzw5PEA8QTxCPEM8RHxIfFb8aDxzvH38f7yaPKS8pz//f//AAH/4w/9D/wP+w/2D/QP6w/UD84Pyg++D6IPlA+QD4gPgQ92D1kPUQ9ND0UPRA89DyIPIA8eDxsPFw8IDs8OjA5fDjcOMQ3IDZ8NlgADAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAH//wAPAAEAAAAAAAAAAAACAAA3OQEAAAAAAQAAAAAAAAAAAAIAADc5AQAAAAABAAAAAAAAAAAAAgAANzkBAAAAAAIAAAAAAtsDbgAbADcAACUUBiMhIiY1NDc+ATc2Mx4BMzI2NzIXHgEXFhUDFAcOAQcGIyInLgEnJjU0Nz4BNzYzMhceARcWAttHMv4YMkgICColJDgjWzU0XCM4JCQqCAiSERE8KCgtLigoOxESEhE7KCguLSgoPBERlz5ZWT45OTpdHh0hKCghHR5dOjk5AfstKCg8ERERETwoKC0uKCg7ERISETsoKAAAAAAEAAAASQO3A24AEAAhADEAQQAAARUUBiMhIiY9ATQ2MyEyFhURFRQGIyEiJj0BNDYzITIWFQEVFAYjISImPQE0N .............
But the goaccess/html Folder is empty
In my config i added enable-panel VISITORS
Some steps more in the right direction ;-)
I have added output /srv/report/index.html in the config - now the log is follwing
2023-03-09T06:34:27.799516396Z Cleaning up resources...
2023-03-09T06:34:29.800392394Z [PARSING /srv/logs/proxy-host-1_access-geo.log] {1} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {3895} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {7992} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {12172} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {16077} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {20053} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {23965} @ {0/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {27900} @ {27900/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {31581} @ {31581/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {35290} @ {35290/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {39063} @ {39063/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {42705} @ {42705/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {46565} @ {46565/s}
[PARSING /srv/logs/proxy-host-1_access-geo.log] {50027} @ {50027/s}
2023-03-09T06:34:29.878107176Z
WebSocket server ready to accept new client connections
But if i will open the webpage, the page is empty.
In the browser console i have following entries:
Content Security Policy: Die Einstellungen der Seite haben das Laden einer Ressource auf inline blockiert ("default-src"). [panel.js:62:22](chrome://global/content/elements/panel.js)
Structured Ingestion ping failure with error: error 4 [PingCentre.jsm:129:15](resource:///modules/PingCentre.jsm)
sendStructuredIngestionPing resource:///modules/PingCentre.jsm:129