haproxy bad OCSP reponse after renew of certificate

[pa77777]

Detailed Description of the Problem

we have problem with OCSP when we renew certificate and then update it in haproxy crt-list (LE certificates)

updating certificate we do like

set ssl cert ${CERT} <<\n$(cat ${CERT}|awk '!/^$/')
commit ssl cert ${CERT}
update ssl ocsp-response ${CERT}

everything is done without any error but

curl https://$DOMAIN -v -I --cert-status
.
.
*  SSL certificate verify ok.
* Could not find certificate ID in OCSP response
* Closing connection 0
curl: (91) Could not find certificate ID in OCSP response
.
.

when we reload haproxy everything is alright, OSCP response is good, but without reload it fails

another solutions is to remove completly cert from haproxy when update it (but OCSP is still in haproxy list of OCSP certs, it's weird), then add it as new, OCSP works as expected

del ssl crt-list ${CRTLIST} ${CERT}
del ssl cert ${CERT}"
new ssl cert ${CERT}
set ssl cert ${CERT}
commit ssl cert ${CERT}
update ssl ocsp-response ${CERT}

curl https://$DOMAIN -v -I --cert-status
.
.
*  SSL certificate verify ok.
* SSL certificate status: good (0)
.
.

but still there is some minimal time when proxy is without that cert, it's problem for us

Expected Behavior

updating certs and regenerating ocsp response does not mess up OCSP reponse to clients

Steps to Reproduce the Behavior

updating certificate we do like

set ssl cert ${CERT} <<\n$(cat ${CERT}|awk '!/^$/')
commit ssl cert ${CERT}
update ssl ocsp-response ${CERT}

everything is done without any error but

curl https://$DOMAIN -v -I --cert-status
returning
*  SSL certificate verify ok.
* Could not find certificate ID in OCSP response
* Closing connection 0
curl: (91) Could not find certificate ID in OCSP response

Do you have any idea what may have caused this?

probably OCSP cert staying in list even if you remove cert and maybe some ID, but from what is showing haproxy from cli it looks like everything is ok

Do you have an idea how to solve the issue?

no, only how to bypass it, but it's not solution

What is your configuration?

bind ipv4@"$IPV4_ADDRESS":443,ipv6@"$IPV6_ADDRESS":443 strict-sni ssl crt /etc/certs alpn h2,http/1.1

Output of haproxy -vv

HAProxy version 2.8.3-86e043a 2023/09/07 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2028.
Known bugs: http://www.haproxy.org/bugs/bugs-2.8.3.html
Running on: Linux 4.18.0-477.27.1.el8_8.x86_64 #1 SMP Wed Sep 20 15:55:39 UTC 2023 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = cc
  CFLAGS  = -O2 -g -Wall -Wextra -Wundef -Wdeclaration-after-statement -Wfatal-errors -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-cast-function-type -Wno-string-plus-int -Wno-atomic-alignment
  OPTIONS = USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_SYSTEMD=1 USE_QUIC=1 USE_PROMEX=1 USE_PCRE2=1 USE_PCRE2_JIT=1
  DEBUG   = -DDEBUG_STRICT -DDEBUG_MEMORY_POOLS

Feature list : -51DEGREES +ACCEPT4 +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ENGINE +EPOLL -EVPORTS +GETADDRINFO -KQUEUE -LIBATOMIC +LIBCRYPT +LINUX_CAP +LINUX_SPLICE +LINUX_TPROXY +LUA +MATH -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL -OPENSSL_WOLFSSL -OT -PCRE +PCRE2 +PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL +PROMEX -PTHREAD_EMULATION +QUIC +RT +SHM_OPEN -SLZ +SSL -STATIC_PCRE -STATIC_PCRE2 +SYSTEMD +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL +ZLIB

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_TGROUPS=16, MAX_THREADS=256, default=3).
Built with OpenSSL version : OpenSSL 1.1.1w+quic  11 Sep 2023
Running on OpenSSL version : OpenSSL 1.1.1w+quic  11 Sep 2023
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.4
Built with the Prometheus exporter as a service
Built with network namespace support.
Built with zlib version : 1.2.11
Running on zlib version : 1.2.11
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.32 2018-09-10
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 8.5.0 20210514 (Red Hat 8.5.0-18)

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
       quic : mode=HTTP  side=FE     mux=QUIC  flags=HTX|NO_UPG|FRAMED
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG

Available services : prometheus-exporter
Available filters :
	[BWLIM] bwlim-in
	[BWLIM] bwlim-out
	[CACHE] cache
	[COMP] compression
	[FCGI] fcgi-app
	[SPOE] spoe
	[TRACE] trace

Last Outputs and Backtraces

No response

Additional Information

No response

[rlebreton]

Thanks for your report, there indeed seems to be lots of unmanaged stuff related to OCSP when you start playing with the SSL cert CLI commands. Some of them are directly linked to the OCSP update but some others pre-date it. I'll try to look into it in order to make it a bit more coherent.

[capflam]

@rlebreton, what is the status of this issue ?

[rlebreton]

The patch already merged should fix this precise issue. We kept a previous OCSP certid instead of reconstructing it when the certificate is changed.

[capflam]

Ok thanks, I'm closing the issue.