zabbix-docker
zabbix-docker copied to clipboard
zabbix
Alpine curl issues in 6.4.16 / 7.0.0 due to Alpine 3.20 upgrade
SUMMARY
Many web-related things (webchecks, Javascript HTTP Client) are throwing this error Failed writing received data to disk/application after upgrading to Zabbix 6.4.16 and 7.0.0.
Both have in common that they are using alpine:3.20 instead of alpine:3.19.
A workaround is to explicitly set the Accept-Encoding: identity header in the webchecks / Javascript HTTP client or to use the Ubuntu container as it is not affected by this issue.
It appears that shipped libcurl is causing issues with this.
- Ubuntu:
libcurl4t64:amd648.5.0-2ubuntu10.1 - Alpine:
libcurl-8.7.1-r0
There are quite a few commits between the 8.5.0 and 8.7.1 release, something must have changed that causes this issue.
OS / ENVIRONMENT / Used docker-compose files
zabbix/zabbix-server-pgsql:alpine-6.4.16
CONFIGURATION
STEPS TO REPRODUCE
- Run server container
zabbix/zabbix-server-pgsql:alpine-6.4.15on Alpine - Create a webcheck
- See it works
- Upgrade to
zabbix/zabbix-server-pgsql:alpine-6.4.16 - See it is showing an error:
Failed writing received data to disk/application - Swap to
zabbix/zabbix-server-pgsql:ubuntu-6.4.16 - Error is gone
EXPECTED RESULTS
- See above
ACTUAL RESULTS
Errors given: `Failed writing received data to disk/application``
RELATED ISSUES
These issues are (potentially) related to this:
- https://support.zabbix.com/browse/ZBX-23702
- https://github.com/curl/curl/pull/12340
I would recommend reporting this issue upstream and downgrading the base image to alpine:3.19 until this issue is resolved as this breaks webchecks, javascript client and notifications.
I vote for pinning the Alpine image before the issue is fixed because it's really annoying. Users would need to change many webchecks to mitigate it so that their monitoring works again, changing the base image is way easier mitigation.
The problem must be fixed in Curl or in Zabbix, not guarantee that the affected version will not be available in other distributions or Alpine 3.19 as well.
The curl issue is linked, and they won't change the library, so Zabbix must work around it on their side. The issue won't be present in Alpine 3.19 as it has curl 8.5.0, while 3.20 got 8.7.1 and edge 8.8.0: minor version in Alpine releases will stay fixed except for edge, which could be updated at any time.
Sticking to Alpine 3.19 is the cheapest and most robust workaround until the Zabbix team fixes the issue.
Unfortunately I can not reproduce the issue. May I ask somebody to share info how to reproduce it? I tried to use https://support.zabbix.com/browse/ZBX-23702, but without success.
Reproduced the issue, on Alpine 3.19 as well:
# docker run -ti alpine:3.19 sh
# apk update && apk add curl
# curl -V
curl 8.5.0 (aarch64-alpine-linux-musl) libcurl/8.5.0 OpenSSL/3.1.5 zlib/1.3.1 brotli/1.1.0 c-ares/1.27.0 libidn2/2.3.4 nghttp2/1.58.0
Release-Date: 2023-12-06
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM SSL threadsafe TLS-SRP UnixSockets
# curl https://www.fust.ch/ --compressed
...
curl: (23) Failed writing received data to disk/application
So the problem exists on 3.19 as well...
If you try to use the following way, does it help: https://wiki.defect.ch/apps/zabbix/template/docker#failed_writing_received_data_to_diskapplication ?
I'm using zabbix/zabbix-server-mysql:latest, which is currently affected by the bug. However, adding RUN wget "https://dl-cdn.alpinelinux.org/alpine/edge/main/$(apk --print-arch)/libcurl-8.8.0-r0.apk" && apk add libcurl-8.8.0-r0.apk && rm libcurl-8.8.0-r0.apk to my local build fixed the issue.
Fix was introduced in https://github.com/curl/curl/commit/b30d694a027eb771c02a3db0dee0ca03ccab7377 which was released as 8.8.0 and was cherry-picked into Debian but not Alpine yet it seems.
Hmm... I tested 8.8.0, but this one still failed:
curl https://www.fust.ch/ --compressed
no?
Please verify that you are using curl 8.8.0 as with the instruction by the link and your comment above you would end up with libcurl 8.8.0 but curl with version from the repo (8.5.0 or 8.7.0).
I can reproduce the issue on website you used for example but not in Zabbix on my website:
/var/lib/zabbix # curl --version
curl 8.8.0 (aarch64-alpine-linux-musl) libcurl/8.8.0 OpenSSL/3.3.1 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 c-ares/1.28.1 libidn2/2.3.7 libpsl/0.21.5 nghttp2/1.62.0
Release-Date: 2024-05-22
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets zstd
/var/lib/zabbix # curl https://www.fust.ch/ --compressed >/dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 112k 0 112k 0 0 1090k 0 --:--:-- --:--:-- --:--:-- 1098k
curl: (23) Failed writing received data to disk/application
Either curl behaves differently than libcurl, or only a portion of the problem is solved. In my case, the issue seems to be related to brotli compression
So if you use web item / web monitoring for https://www.fust.ch/ everything is fine on Zabbix side?
With www.fust.ch, Zabbix with the libcurl 8.8.0 still gives the error Failed writing received data to disk/application. I do not know what the difference is with my host.
/var/lib/zabbix # curl https://www.fust.ch/ --compressed > /dev/null ; echo $?
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
94 126k 94 120k 0 0 1322k 0 --:--:-- --:--:-- --:--:-- 1333k
curl: (23) Failed writing received data to disk/application
23
/var/lib/zabbix # curl https://favor-group.ru/ --compressed > /dev/null ; echo $
?
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 40897 0 40897 0 0 114k 0 --:--:-- --:--:-- --:--:-- 115k
0
Encoding:
/var/lib/zabbix # curl https://www.fust.ch/ --compressed -sI | grep encoding
content-encoding: gzip
With my host which was not working properly prior to curl update:
/var/lib/zabbix # curl https://favor-group.ru/ --compressed -sI | grep encoding
content-encoding: br
Seems like the problem is fixed for brotli compression but still present under some other conditions. And in case you are in any way responsible for fust.ch, it's HTTPS certificate seems to expire in 4 days.
Looks like all Alpine versions now have libcurl 8.9.0. Now I do not see any issues with https://favor-group.ru/, any other hosts?
It works for me without any additional tinkering with the libcurl version, thank you for the update!