BUG: broken json for backup upstream for nginx 1.27.X (after upstream resolver public added)

[gleb-shchavlev]

module version checks

• [x] I have checked that this issue has not already been reported.

Issue Description

VTS build from master. nginx 1.27.5

config:

http {
  ...
  upstream resolver.local {
    zone clients 10m;
    server 127.0.0.1;
  }
  ...
  server {
    listen 80;
    location /vts {
      gzip on;
      gzip_types application/json;
      vhost_traffic_status_display;
      vhost_traffic_status_display_format html;
      vhost_traffic_status_bypass_stats on;
    }
  }
  upstream test {
    zone clients;
    server google.com:443 resolve;
    server www.google.com:443 backup resolve;
  }
  server {
    listen 777;
    location / {
      proxy_pass https://test;
    }
  }
  ...
}

Open 127.0.0.1/vts, all fine before console command: curl 127.0.0.1:777

After got error in browser console: VM141:1 Uncaught SyntaxError: Bad control character in string literal in JSON at position 16605 (line 1 column 16606) at JSON.parse (<anonymous>) at request.onload (vts:828:55)

In json block with server name for backup server is broken: { "server": "",

The issue appears:

1. If there is nginx with resolve command in upstream support
2. If there is a backup server in the upstream block

Expected Behavior

Json does not broken.

Installed Versions

$ nginx -V nginx version: nginx/1.27.5 built by gcc 11.5.0 20240719 (Red Hat 11.5.0-5) (GCC) built with OpenSSL 3.3.3 11 Feb 2025 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-openssl=../openssl-openssl-3.3.3 --with-openssl-opt='--release enable-rc5 enable-ssl3 enable-ssl3-method enable-weak-ssl-ciphers enable-ktls -DOPENSSL_TLS_SECURITY_LEVEL=1' --with-http_realip_module --add-module=../nginx-module-vts --add-module=../nginx-vod-module-master --add-module=../headers-more-nginx-module-0.34 --add-module=../lua-nginx-module-master --add-module=../ngx_brotli-master --add-module=../encrypted-session-nginx-module-0.09 --add-module=../ngx_http_geoip2_module-3.4 --add-module=../redis2-nginx-module-0.15 --add-module=../set-misc-nginx-module-0.33 --add-module=../ngx_http_substitutions_filter_module-master --add-module=../nginx_accept_language_module-master --add-module=../njs-0.8.9/nginx --with-cc-opt='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64-v2 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'

Reproducible Example nginx.conf

No response

[u5surf]

@gleb-shchavlev Thanks reporting! We could fix the invaild metrics outputs. But we could only output peers which is resolved at that moment, hasn't included the past resolved peer's metrics. We consider that it should output whole peers which has already resolved at least once when we use upstream dynamic resolve which introduced from nginx 1.27.3. We should reconsider that issue. The whole of metrics seems to been recorded already on rbtree’s nodes.

[gleb-shchavlev]

Thanks for the response, I'm glad the project is alive! Is there any idea how long fix will take?

[u5surf]

@gleb-shchavlev We could not gurantee when it fixed completely, becaues each we have another proper ocupations. At least an invalid view must be fixed soon. But a whole peer's output in dynamic upstream resolution issue may be fixed more long time because we should not only fix entire of upstreams output operations, but also might be fixed recording the metrics. So, we cannot guess how long properly.

[gleb-shchavlev]

We started using your patch in pre-production environment. It looks like everything works correctly.

[u5surf]

@gleb-shchavlev Thanks. Could you try #322 in your pre-production environment? It can be retrieved the past resolved metrics.

[gleb-shchavlev]

Thanks for patch. Merged and deployed to pre-production environment.