vozlt
Error in `nginx: worker process': corrupted size vs. prev_size: 0x000055bacb0056d0
Periodically crashes worker-processes vts-config: vhost_traffic_status_zone shared:vhost_traffic_status:64m; vhost_traffic_status_filter_by_set_key $status $server_name:$server_port; server { listen 127.0.0.1:8888; server_name status; location /nginx_status { stub_status on; } location /extended_status { vhost_traffic_status_display; vhost_traffic_status_display_format json; } } 2019/11/26 11:26:54 [alert] 14921#14921: worker process 5452 exited on signal 6 *** Error in `nginx: worker process': corrupted size vs. prev_size: 0x000055ac87e8cad0 *** ======= Backtrace: ========= /lib64/libc.so.6(+0x7f7c4)[0x7f1031b577c4] /lib64/libc.so.6(+0x8309e)[0x7f1031b5b09e] /lib64/libc.so.6(__libc_malloc+0x4c)[0x7f1031b5dadc] /lib64/libc.so.6(posix_memalign+0x4c)[0x7f1031b6004c] nginx: worker process(ngx_memalign+0x2f)[0x55ac85ce48bf] nginx: worker process(+0xb6312)[0x55ac85cc3312] nginx: worker process(ngx_pcalloc+0xe)[0x55ac85cc37fe] /etc/nginx/modules/ngx_http_vhost_traffic_status_module.so(ngx_http_vhost_traffic_status_escape_json_pool+0x1e)[0x7f103148f8fe] /etc/nginx/modules/ngx_http_vhost_traffic_status_module.so(ngx_http_vhost_traffic_status_display_set_upstream_node+0x59)[0x7f1031495e79] /etc/nginx/modules/ngx_http_vhost_traffic_status_module.so(ngx_http_vhost_traffic_status_display_set_upstream_group+0x555)[0x7f1031496aa5] /etc/nginx/modules/ngx_http_vhost_traffic_status_module.so(ngx_http_vhost_traffic_status_display_set+0x17f)[0x7f103149709f] /etc/nginx/modules/ngx_http_vhost_traffic_status_module.so(+0xb96b)[0x7f103149496b] nginx: worker process(ngx_http_core_content_phase+0x31)[0x55ac85cfd9a1] nginx: worker process(ngx_http_core_run_phases+0x2d)[0x55ac85cf7fcd] nginx: worker process(+0xf66e1)[0x55ac85d036e1] nginx: worker process(+0xf6a5e)[0x55ac85d03a5e] nginx: worker process(+0xddd07)[0x55ac85cead07] nginx: worker process(ngx_process_events_and_timers+0x6a)[0x55ac85ce12ea] nginx: worker process(+0xdbfb1)[0x55ac85ce8fb1] nginx: worker process(ngx_spawn_process+0x18b)[0x55ac85ce745b] nginx: worker process(ngx_master_process_cycle+0x73a)[0x55ac85ce9eea] nginx: worker process(main+0x8bf)[0x55ac85cc0d7f] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f1031afa505] nginx: worker process(+0xb40fe)[0x55ac85cc10fe] ======= Memory map: ======== 55ac85c0d000-55ac86014000 r-xp 00000000 103:01 17578781 /usr/sbin/nginx 55ac86213000-55ac86245000 r--p 00406000 103:01 17578781 /usr/sbin/nginx 55ac86245000-55ac8626b000 rw-p 00438000 103:01 17578781 /usr/sbin/nginx 55ac8626b000-55ac8628f000 rw-p 00000000 00:00 0 55ac87d26000-55ac87dc8000 rw-p 00000000 00:00 0 [heap] 55ac87dc8000-55ac87f50000 rw-p 00000000 00:00 0 [heap] 7f1028000000-7f1028021000 rw-p 00000000 00:00 0 7f1028021000-7f102c000000 ---p 00000000 00:00 0 7f102d273000-7f102d288000 r-xp 00000000 103:01 17578233 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f102d288000-7f102d487000 ---p 00015000 103:01 17578233 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f102d487000-7f102d488000 r--p 00014000 103:01 17578233 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f102d488000-7f102d489000 rw-p 00015000 103:01 17578233 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f102d489000-7f1031489000 rw-s 00000000 00:04 618913036 /dev/zero (deleted) 7f1031489000-7f10314a0000 r-xp 00000000 103:01 17957501 /usr/lib64/nginx/modules/ngx_http_vhost_traffic_status_module.so 7f10314a0000-7f10316a0000 ---p 00017000 103:01 17957501 /usr/lib64/nginx/modules/ngx_http_vhost_traffic_status_module.so 7f10316a0000-7f10316a1000 r--p 00017000 103:01 17957501 /usr/lib64/nginx/modules/ngx_http_vhost_traffic_status_module.so 7f10316a1000-7f10316c2000 rw-p 00018000 103:01 17957501 /usr/lib64/nginx/modules/ngx_http_vhost_traffic_status_module.so 7f10316c2000-7f10316ce000 r-xp 00000000 103:01 17578354 /usr/lib64/libnss_files-2.17.so 7f10316ce000-7f10318cd000 ---p 0000c000 103:01 17578354 /usr/lib64/libnss_files-2.17.so 7f10318cd000-7f10318ce000 r--p 0000b000 103:01 17578354 /usr/lib64/libnss_files-2.17.so 7f10318ce000-7f10318cf000 rw-p 0000c000 103:01 17578354 /usr/lib64/libnss_files-2.17.so 7f10318cf000-7f10318d5000 rw-p 00000000 00:00 0 7f10318d5000-7f10318d7000 r-xp 00000000 103:01 17578226 /usr/lib64/libfreebl3.so 7f10318d7000-7f1031ad6000 ---p 00002000 103:01 17578226 /usr/lib64/libfreebl3.so 7f1031ad6000-7f1031ad7000 r--p 00001000 103:01 17578226 /usr/lib64/libfreebl3.so 7f1031ad7000-7f1031ad8000 rw-p 00002000 103:01 17578226 /usr/lib64/libfreebl3.so 7f1031ad8000-7f1031c9b000 r-xp 00000000 103:01 17578156 /usr/lib64/libc-2.17.so 7f1031c9b000-7f1031e9b000 ---p 001c3000 103:01 17578156 /usr/lib64/libc-2.17.so 7f1031e9b000-7f1031e9f000 r--p 001c3000 103:01 17578156 /usr/lib64/libc-2.17.so 7f1031e9f000-7f1031ea1000 rw-p 001c7000 103:01 17578156 /usr/lib64/libc-2.17.so 7f1031ea1000-7f1031ea6000 rw-p 00000000 00:00 0 7f1031ea6000-7f1031ebb000 r-xp 00000000 103:01 17578549 /usr/lib64/libz.so.1.2.7 7f1031ebb000-7f10320ba000 ---p 00015000 103:01 17578549 /usr/lib64/libz.so.1.2.7 7f10320ba000-7f10320bb000 r--p 00014000 103:01 17578549 /usr/lib64/libz.so.1.2.7 7f10320bb000-7f10320bc000 rw-p 00015000 103:01 17578549 /usr/lib64/libz.so.1.2.7 7f10320bc000-7f103211c000 r-xp 00000000 103:01 17578386 /usr/lib64/libpcre.so.1.2.0 7f103211c000-7f103231c000 ---p 00060000 103:01 17578386 /usr/lib64/libpcre.so.1.2.0 7f103231c000-7f103231d000 r--p 00060000 103:01 17578386 /usr/lib64/libpcre.so.1.2.0 7f103231d000-7f103231e000 rw-p 00061000 103:01 17578386 /usr/lib64/libpcre.so.1.2.0 7f103231e000-7f1032326000 r-xp 00000000 103:01 17578173 /usr/lib64/libcrypt-2.17.so 7f1032326000-7f1032525000 ---p 00008000 103:01 17578173 /usr/lib64/libcrypt-2.17.so 7f1032525000-7f1032526000 r--p 00007000 103:01 17578173 /usr/lib64/libcrypt-2.17.so 7f1032526000-7f1032527000 rw-p 00008000 103:01 17578173 /usr/lib64/libcrypt-2.17.so 7f1032527000-7f1032555000 rw-p 00000000 00:00 0 7f1032555000-7f103256c000 r-xp 00000000 103:01 17578404 /usr/lib64/libpthread-2.17.so 7f103256c000-7f103276b000 ---p 00017000 103:01 17578404 /usr/lib64/libpthread-2.17.so 7f103276b000-7f103276c000 r--p 00016000 103:01 17578404 /usr/lib64/libpthread-2.17.so 7f103276c000-7f103276d000 rw-p 00017000 103:01 17578404 /usr/lib64/libpthread-2.17.so 7f103276d000-7f1032771000 rw-p 00000000 00:00 0 7f1032771000-7f1032773000 r-xp 00000000 103:01 17578194 /usr/lib64/libdl-2.17.so 7f1032773000-7f1032973000 ---p 00002000 103:01 17578194 /usr/lib64/libdl-2.17.so 7f1032973000-7f1032974000 r--p 00002000 103:01 17578194 /usr/lib64/libdl-2.17.so 7f1032974000-7f1032975000 rw-p 00003000 103:01 17578194 /usr/lib64/libdl-2.17.so 7f1032975000-7f1032997000 r-xp 00000000 103:01 17578081 /usr/lib64/ld-2.17.so 7f1032a8c000-7f1032b85000 rw-p 00000000 00:00 0 7f1032b85000-7f1032b8a000 rw-p 00000000 00:00 0 7f1032b92000-7f1032b93000 rw-p 00000000 00:00 0 7f1032b93000-7f1032b94000 rw-s 00000000 00:0b 648909370 /[aio] (deleted) 7f1032b94000-7f1032b95000 rw-s 00000000 00:04 618913040 /dev/zero (deleted) 7f1032b95000-7f1032b96000 rw-p 00000000 00:00 0 7f1032b96000-7f1032b97000 r--p 00021000 103:01 17578081 /usr/lib64/ld-2.17.so 7f1032b97000-7f1032b98000 rw-p 00022000 103:01 17578081 /usr/lib64/ld-2.17.so 7f1032b98000-7f1032b99000 rw-p 00000000 00:00 0 7ffdffbc4000-7ffdffbe5000 rw-p 00000000 00:00 0 [stack] 7ffdffbf2000-7ffdffbf4000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] 2019/11/26 11:26:55 [alert] 14921#14921: worker process 5764 exited on signal 6 2019/11/26 11:26:55 [alert] 14921#14921: shared memory zone "vhost_traffic_status" was locked by 5764
nginx -V nginx version: nginx/1.16.1 built by gcc 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) built with OpenSSL 1.1.1d 10 Sep 2019 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-openssl=openssl-1.1.1d --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie' --with-zlib-opt=-fPIC --with-openssl-opt='no-ssl2 no-ssl3 -fPIC enable-tls1_3'
@dimsua Does it still happen with the latest version of Nginx(>=1.18.0) and VTS? Latest commit: https://github.com/vozlt/nginx-module-vts/commit/91bdb146e45e612d1ce0b257bcda638848367d84
