There is a bug in the base32+src stick-table

[zino7825]

Detailed Description of the Problem

I use the stick table in backend as base32+src tracking. It hasn't been a problem for a week.

There was a limit over based on the base32+src today, little later, the same path requests began to blocked regardless of ip

All "/Articlexxx" path requests were blocked. ( red block )

Expected Behavior

Since I use the base32+src tracking, it should have blocked only requests that exceed limit based on ip+path, not the entire ip.

Steps to Reproduce the Behavior

I have no ideas. I'm trying sticktable without blocking for reproducing.

Do you have any idea what may have caused this?

No response

Do you have an idea how to solve the issue?

No response

What is your configuration?

backend xxx

stick-table type string len 128 size 1m expire 3s store http_req_rate(3s)
http-request track-sc1 base32+src
http-request set-var(req.path_limit) path,map_beg(/etc/acl/xxx/rates.map,100)

http-request set-var(req.path_rate) sc_http_req_rate(1)
acl is_ratelimit var(req.path_limit),sub(req.path_rate) -m int lt 0

http-request silent-drop if is_ratelimit

server xxx

Output of haproxy -vv

HAProxy version 2.5.5 2022/03/14 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2023.
Known bugs: http://www.haproxy.org/bugs/bugs-2.5.5.html
Running on: Linux 3.10.0-957.21.3.el7.x86_64 #1 SMP Tue Jun 18 16:35:19 UTC 2019 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = native
  CC      = cc
  CFLAGS  = -O2 -march=native -g -Wall -Wextra -Wundef -Wdeclaration-after-statement -fwrapv -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wtype-limits -DHAPROXY_TARGET_VERSION=250 -DTLS_TICKETS_NO=4
  OPTIONS = USE_PCRE=1 USE_PCRE_JIT=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1
  DEBUG   =

Feature list : +EPOLL -KQUEUE +NETFILTER +PCRE +PCRE_JIT -PCRE2 -PCRE2_JIT +POLL +THREAD +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H +GETADDRINFO +OPENSSL +LUA +ACCEPT4 -CLOSEFROM +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL -PROCCTL +THREAD_DUMP -EVPORTS -OT -QUIC -PROMEX -MEMORY_PROFILING

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=20).
Built with OpenSSL version : OpenSSL 1.1.1o  3 May 2022
Running on OpenSSL version : OpenSSL 1.1.1o  3 May 2022
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.5
Built with network namespace support.
Built with Naver SSL Client Hello request capture. version: RB-1.0.1:113175
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Support for malloc_trim() is enabled.
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE version : 8.43 2019-02-23
Running on PCRE version : 8.43 2019-02-23
PCRE library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 4.8.5 20150623 (Red Hat 4.8.5-11)

Last Outputs and Backtraces

No response

Additional Information

No response

[zino7825]

I found the problem. If the string type is used, only one table raw is maintained.

## add acl : path_beg /Articlexxx for traking 
This stick-table store type is string
[root@xxxx conf.d]# hcli show table {backend_name}
# table: {backend_name}, type: string, size:1048576, used:1
0x7f0bfc09c018: key=r use=4 exp=2852 http_req_rate(3000)=21 http_err_rate(3000)=0

When i use store type binary 
[root@xxx conf.d]# hcli show table {backend_name}
# table: {backend_name}, type: binary, size:1048576, used:37
0x7fbcd4687c98: key=7200E43201D61725000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 use=0 exp=1246 http_req_rate(3000)=1 http_err_rate(3000)=0
0x7fbc942f5d68: key=7200E43201F74FC6000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 use=0 exp=498 http_req_rate(3000)=1 http_err_rate(3000)=0
0x7fbcc4826c68: key=7200E43201FFE352000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 use=0 exp=2140 http_req_rate(3000)=1 http_err_rate(3000)=0
0x7fbd440b8c58: key=7200E432277CBB9A000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 use=0 exp=753 http_req_rate(3000)=1 http_err_rate(3000)=0
0x7fbcdc72be88: key=7200E43231AB1A65000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 use=0 exp=1913 http_req_rate(3000)=1 http_err_rate(3000)=0
0x7fbcd04f7e08: key=7200E4323A966D9E000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 use=0 exp=929 http_req_rate(3000)=1 http_err_rate(3000)=0
.
.
.

[wtarreau]

Ah I think I got it. The zeroes that are present in the hash are shortening the key when converted to string and you can quickly enter into trouble because multiple entries may match the same key.

There's not much else we can do to improve the situation here given that everything there is valid, it's just the combination that makes little sense even if that can easily happen by mistake.

I'm marking as "works as designed", feel free to close unless you want to add anything. Thanks!

[zino7825]

@wtarreau What you mean is that the "Base32+src and storage type string" combination can be a problem?

What's unusual is that there's no problem with other domains, but only this domain and this path has problems.

[wtarreau]

The problem is that if some URI hashes produce a nul character, it will result in a truncated string. And if you're unlucky, maybe the "/" for a certain domain starts with a zero, in which case all URLs map to a single entry, which explains that you can easily end up with 100% failures once nobody can reach the first page.

[capflam]

I guess we can close it now