netdata
netdata copied to clipboard
netdata
[Bug]: haproxy metrics do not show up in the dashboard
Bug description
My /etc/netdata/python.d/haproxy.conf:
socket: '/var/run/haproxy/admin.sock'
If I run the following command:
/usr/libexec/netdata/plugins.d$ ./python.d.plugin haproxy
Then I see metrics scrolling by, so I assume the collector is working. But the dashboard (both on the local netdata instance, and in Netdata Cloud) does not show any of the metrics except "haproxy.idle".
This used to work but I guess broke during one of the upgrades.
PS. I also noticed the Netdata docs for monitoring haproxy now give examples using the go.d plugin, not the python.d plugin. Is the python.d plugin deprecated and that's why its metrics do not show in the dashboard any more? The go.d plugin does not seem to support reading haproxy metrics from a socket so it is unfortunately not a drop-in replacement.
Expected behavior
haproxy metrics show up in the dashboard
Steps to reproduce
- Set up
/etc/netdata/python.d/haproxy.conf - Restart netdata agent
Installation method
manual setup of official DEB/RPM packages
System info
Linux myhost 6.8.0-79-generic #79-Ubuntu SMP PREEMPT_DYNAMIC Tue Aug 12 14:42:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
/etc/lsb-release:DISTRIB_ID=Ubuntu
/etc/lsb-release:DISTRIB_RELEASE=24.04
/etc/lsb-release:DISTRIB_CODENAME=noble
/etc/lsb-release:DISTRIB_DESCRIPTION="Ubuntu 24.04.3 LTS"
/etc/os-release:PRETTY_NAME="Ubuntu 24.04.3 LTS"
/etc/os-release:NAME="Ubuntu"
/etc/os-release:VERSION_ID="24.04"
/etc/os-release:VERSION="24.04.3 LTS (Noble Numbat)"
/etc/os-release:VERSION_CODENAME=noble
/etc/os-release:ID=ubuntu
/etc/os-release:ID_LIKE=debian
/etc/os-release:UBUNTU_CODENAME=noble
/etc/os-release:LOGO=ubuntu-logo
Netdata build info
Packaging:
Netdata Version ____________________________________________ : v2.6.3
Installation Type __________________________________________ : binpkg-deb
Package Architecture _______________________________________ : x86_64
Package Distro _____________________________________________ : ubuntu 24.04
Configure Options __________________________________________ : cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_C_STANDARD=11 -DCMAKE_CXX_STANDARD=14 -DBUILD_SHARED_LIBS= -DCMAKE_C_FLAGS=' -fexceptions -fexceptions -fno-omit-frame-pointer -funwind-tables -fasynchronous-unwind-tables' -DCMAKE_CXX_FLAGS=' -fexceptions -fexceptions -fno-omit-frame-pointer -funwind-tables -fasynchronous-unwind-tables' -DCMAKE_COMPILE_DEFINITIONS='_GNU_SOURCE' -DCMAKE_EXE_LINKER_FLAGS=' -fexceptions -fexceptions -rdynamic' -DCMAKE_SHARED_LINKER_FLAGS=''
Default Directories:
User Configurations ________________________________________ : /etc/netdata
Stock Configurations _______________________________________ : /usr/lib/netdata/conf.d
Ephemeral Databases (metrics data, metadata) _______________ : /var/cache/netdata
Permanent Databases ________________________________________ : /var/lib/netdata
Plugins ____________________________________________________ : /usr/libexec/netdata/plugins.d
Static Web Files ___________________________________________ : /usr/share/netdata/web
Log Files __________________________________________________ : /var/log/netdata
Lock Files _________________________________________________ : /var/lib/netdata/lock
Home _______________________________________________________ : /var/lib/netdata
Operating System:
Kernel _____________________________________________________ : Linux
Kernel Version _____________________________________________ : 6.8.0-79-generic
Operating System ___________________________________________ : Ubuntu
Operating System ID ________________________________________ : ubuntu
Operating System ID Like ___________________________________ : debian
Operating System Version ___________________________________ : 24.04.3 LTS (Noble Numbat)
Operating System Version ID ________________________________ : none
Detection __________________________________________________ : /etc/os-release
Hardware:
CPU Cores __________________________________________________ : 12
CPU Frequency ______________________________________________ : 3600000000
RAM Bytes __________________________________________________ : 67336523776
Disk Capacity ______________________________________________ : 1024220381184
CPU Architecture ___________________________________________ : x86_64
Virtualization Technology __________________________________ : none
Virtualization Detection ___________________________________ : systemd-detect-virt
Container:
Container __________________________________________________ : none
Container Detection ________________________________________ : systemd-detect-virt
Container Orchestrator _____________________________________ : none
Container Operating System _________________________________ : none
Container Operating System ID ______________________________ : none
Container Operating System ID Like _________________________ : none
Container Operating System Version _________________________ : none
Container Operating System Version ID ______________________ : none
Container Operating System Detection _______________________ : none
Features:
Built For __________________________________________________ : Linux
Netdata Cloud ______________________________________________ : YES
Health (trigger alerts and send notifications) _____________ : YES
Streaming (stream metrics to parent Netdata servers) _______ : YES
Back-filling (of higher database tiers) ____________________ : YES
Replication (fill the gaps of parent Netdata servers) ______ : YES
Streaming and Replication Compression ______________________ : YES (zstd lz4 gzip)
Contexts (index all active and archived metrics) ___________ : YES
Tiering (multiple dbs with different metrics resolution) ___ : YES (5)
Machine Learning ___________________________________________ : YES
Memory Allocator ___________________________________________ : system
Database Engines:
dbengine (compression) _____________________________________ : YES (zstd lz4)
alloc ______________________________________________________ : YES
ram ________________________________________________________ : YES
none _______________________________________________________ : YES
Connectivity Capabilities:
ACLK (Agent-Cloud Link: MQTT over WebSockets over TLS) _____ : YES
static (Netdata internal web server) _______________________ : YES
h2o (web server) ___________________________________________ : NO
WebRTC (experimental) ______________________________________ : NO
Native HTTPS (TLS Support) _________________________________ : YES
TLS Host Verification ______________________________________ : YES
Libraries:
LZ4 (extremely fast lossless compression algorithm) ________ : YES
ZSTD (fast, lossless compression algorithm) ________________ : YES
zlib (lossless data-compression library) ___________________ : YES
Brotli (generic-purpose lossless compression algorithm) ____ : NO
protobuf (platform-neutral data serialization protocol) ____ : YES (system)
OpenSSL (cryptography) _____________________________________ : YES
libdatachannel (stand-alone WebRTC data channels) __________ : NO
JSON-C (lightweight JSON manipulation) _____________________ : YES
libcap (Linux capabilities system operations) ______________ : NO
libcrypto (cryptographic functions) ________________________ : YES
libyaml (library for parsing and emitting YAML) ____________ : YES
libmnl (library for working with netfilter) ________________ : YES
stacktraces (library for getting stack traces) _____________ : libbacktrace (mmap, threads, data)
Plugins:
apps (monitor processes) ___________________________________ : YES
cgroups (monitor containers and VMs) _______________________ : YES
cgroup-network (associate interfaces to CGROUPS) ___________ : YES
proc (monitor Linux systems) _______________________________ : YES
tc (monitor Linux network QoS) _____________________________ : YES
diskspace (monitor Linux mount points) _____________________ : YES
freebsd (monitor FreeBSD systems) __________________________ : NO
macos (monitor MacOS systems) ______________________________ : NO
windows (monitor Windows systems) __________________________ : NO
statsd (collect custom application metrics) ________________ : YES
timex (check system clock synchronization) _________________ : YES
idlejitter (check system latency and jitter) _______________ : YES
bash (support shell data collection jobs - charts.d) _______ : YES
debugfs (kernel debugging metrics) _________________________ : YES
cups (monitor printers and print jobs) _____________________ : YES
ebpf (monitor system calls) ________________________________ : YES
freeipmi (monitor enterprise server H/W) ___________________ : YES
network-viewer (monitor TCP/UDP IPv4/6 sockets) ____________ : YES
systemd-journal (monitor journal logs) _____________________ : YES
windows-events (monitor Windows events) ____________________ : NO
nfacct (gather netfilter accounting) _______________________ : YES
perf (collect kernel performance events) ___________________ : YES
slabinfo (monitor kernel object caching) ___________________ : YES
Xen ________________________________________________________ : YES
Xen VBD Error Tracking _____________________________________ : NO
Exporters:
AWS Kinesis ________________________________________________ : NO
GCP PubSub _________________________________________________ : NO
MongoDB ____________________________________________________ : YES
Prometheus (OpenMetrics) Exporter __________________________ : YES
Prometheus Remote Write ____________________________________ : YES
Graphite ___________________________________________________ : YES
Graphite HTTP / HTTPS ______________________________________ : YES
JSON _______________________________________________________ : YES
JSON HTTP / HTTPS __________________________________________ : YES
OpenTSDB ___________________________________________________ : YES
OpenTSDB HTTP / HTTPS ______________________________________ : YES
All Metrics API ____________________________________________ : YES
Shell (use metrics in shell scripts) _______________________ : YES
Debug/Developer Features:
Trace All Netdata Allocations (with charts) ________________ : NO
Developer Mode (more runtime checks, slower) _______________ : NO
Runtime Information:
Profile ____________________________________________________ : standalone
Stream Parent (accept data from Children) __________________ : NO
Stream Child (send data to a Parent) _______________________ : NO
Total System Memory ________________________________________ : 67336523776
Available System Memory ____________________________________ : 65784475648
Additional info
No response
I enabled collector logs and do see the haproxy collector collecting metrics:
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : attempting DGRAM unix socket "/var/run/haproxy/admin.sock"
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : set socket connect timeout to: 2.0
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : Failed to connect DGRAM unix socket "/var/run/haproxy/admin.sock": [Errno 91] Protocol wrong type for socket
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : attempting STREAM unix socket "/var/run/haproxy/admin.sock"
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : set socket connect timeout to: 2.0
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : connected STREAM unix socket "/var/run/haproxy/admin.sock"
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : set socket write timeout to: 2.0
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : sending request: b'show info\n'
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : receiving response
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : set socket read timeout to: 2.0
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : received data
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : receiving response
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : set socket read timeout to: 2.0
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : server closed the connection
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : closing socket
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : final response: Name: HAProxy
Version: 3.0.12-1ppa1~noble
Release_date: 2025/10/03
Nbthread: 12
Nbproc: 1
Process_num: 1
Pid: 1424891
Uptime: 0d 5h58m11s
Uptime_sec: 21491
Memmax_MB: 0
PoolAlloc_MB: 7
PoolUsed_MB: 7
PoolFailed: 0
Ulimit-n: 100094
Maxsock: 100094
Maxconn: 50000
Hard_maxconn: 50000
CurrConns: 293
CumConns: 5055451
CumReq: 5156603
MaxSslConns: 0
CurrSslConns: 288
CumSslConns: 4877713
Maxpipes: 0
PipesUsed: 0
PipesFree: 0
ConnRate: 150
ConnRateLimit: 0
MaxConnRate: 2929
SessRate: 150
SessRateLimit: 0
MaxSessRate: 2928
SslRate: 147
SslRateLimit: 0
MaxSslRate: 2904
SslFrontendKeyRate: 147
SslFrontendMaxKeyRate: 2888
SslFrontendSessionReuse_pct: 0
SslBackendKeyRate: 0
SslBackendMaxKeyRate: 0
SslCacheLookups: 714804
SslCacheMisses: 394289
CompressBpsIn: 0
CompressBpsOut: 0
CompressBpsRateLim: 0
Tasks: 360
Run_queue: 1
Idle_pct: 98
node: lb2
Stopping: 0
Jobs: 305
Unstoppable Jobs: 1
Listeners: 10
ActivePeers: 0
ConnectedPeers: 0
DroppedLogs: 0
BusyPolling: 0
FailedResolutions: 0
TotalBytesOut: 19147067042
TotalSplicedBytesOut: 0
BytesOutRate: 636640
DebugCommandsIssued: 0
CumRecvLogs: 0
Build info: 3.0.12-1ppa1~noble
Memmax_bytes: 0
PoolAlloc_bytes: 7350464
PoolUsed_bytes: 7350464
Start_time_sec: 1762249874
Tainted: 0
TotalWarnings: 0
MaxconnReached: 0
BootTime_ms: 97
Niced_tasks: 0
CurrStreams: 7
CumStreams: 5124155
BlockedTrafficWarnings: 0
2025-11-04 16:49:25: python.d DEBUG: haproxy[via_socket] : update => [OK] (elapsed time: 3, failed retries in a row: 0)
But still only one graph in the web UI.
Ah, oops, haproxy metrics do in fact show up in the web UI. What got me confused is that most haproxy graphs are below the netdata graphs: