plugins icon indicating copy to clipboard operation
plugins copied to clipboard
verified publisher icon opnsense

www/squid: recent segmentation faults in squid binary

Open Kornexl opened this issue 1 year ago • 42 comments

Important notices Before you add a new report, we ask you kindly to acknowledge the following:

  • [x] I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
  • [x] I have searched the existing issues, open and closed, and I'm convinced that mine is new.
  • [x] The title contains the plugin to which this issue belongs

Describe the bug restarting the service produces a "Segmentation fault"

Tip: to validate your setup was working with the previous version, use opnsense-revert (https://docs.opnsense.org/manual/opnsense_tools.html#opnsense-revert)

To Reproduce

  • change a value in Services - squid web proxy
  • apply changes
  • restart services
  • a window with a last line "segmentation fault" pops up

Expected behavior The service should restart with the new config; the service runs without change

Screenshots If applicable, add screenshots to help explain your problem.

Relevant log files If applicable, information from log files supporting your claim.

Additional context Add any other context about the problem here. running "service squid restart" in a shell produces a similar sequence Screenshot 2024-02-19 190453 Screenshot 2024-02-19 191427

Environment Software version used and hardware type if relevant. e.g.:

os-squid 1.0 OPNsense 24.1.1(amd64). on virtualbox vm

Kornexl avatar Feb 19 '24 19:02 Kornexl

if i enter squid -k parse in a shell: Screenshot 2024-02-19 210158

Kornexl avatar Feb 19 '24 20:02 Kornexl

I'm having the same issue. Plugin os-squid 1.0 and OPNsense 24.1.1(amd64) as well.

But the plugin works anyway.

Irrational-NX avatar Feb 20 '24 09:02 Irrational-NX

Confirmed, have the same issue in a production environment. image

dblanque avatar Feb 28 '24 19:02 dblanque

You can try your luck with the newer squid version 6.7 from the snapshots:

# opnsense-revert -z squid

(run without "-z" to revert back to the stable one)

Cheers, Franco

fichtner avatar Feb 28 '24 19:02 fichtner

Hi Fichtner, does this require a restart? Installing Squid 6.7 did not affect the problem whatsoever.

dblanque avatar Feb 28 '24 19:02 dblanque

But it’s the squid process segfaulting?

fichtner avatar Feb 28 '24 19:02 fichtner

Squid itself is running (from system startup), but the service does not apply new configurations or is able to be restarted. The following error arises when attempting to parse the config (whether from the GUI or the console).

This worked before version 24.1, so I might revert back from it in the mean time.

root@opnsense-borde:~ # squid -k parse
2024/02/28 16:56:04| Processing Configuration File: /usr/local/etc/squid/squid.conf (depth 0)
2024/02/28 16:56:04| Processing: http_port 127.0.0.1:3128 intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
2024/02/28 16:56:04| Starting Authentication on port 127.0.0.1:3128
2024/02/28 16:56:04| Disabling Authentication on port 127.0.0.1:3128 (interception enabled)
2024/02/28 16:56:04| Processing: http_port [::1]:3128 intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
2024/02/28 16:56:04| Starting Authentication on port [::1]:3128
2024/02/28 16:56:04| Disabling Authentication on port [::1]:3128 (interception enabled)
2024/02/28 16:56:04| Processing: https_port 127.0.0.1:3129 intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
2024/02/28 16:56:04| Starting Authentication on port 127.0.0.1:3129
2024/02/28 16:56:04| Disabling Authentication on port 127.0.0.1:3129 (interception enabled)
2024/02/28 16:56:04| Processing: https_port [::1]:3129 intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
2024/02/28 16:56:04| Starting Authentication on port [::1]:3129
2024/02/28 16:56:04| Disabling Authentication on port [::1]:3129 (interception enabled)
2024/02/28 16:56:04| Processing: http_port 10.100.0.3:3128  ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
2024/02/28 16:56:04| Processing: sslcrtd_program /usr/local/libexec/squid/security_file_certgen -s /var/squid/ssl_crtd -M 8MB
2024/02/28 16:56:04| Processing: sslcrtd_children 5
2024/02/28 16:56:04| Processing: tls_outgoing_options options=NO_TLSv1 cipher=HIGH:MEDIUM:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS
2024/02/28 16:56:04| Processing: acl bump_step1 at_step SslBump1
2024/02/28 16:56:04| Processing: acl bump_step2 at_step SslBump2
2024/02/28 16:56:04| Processing: acl bump_step3 at_step SslBump3
2024/02/28 16:56:04| Processing: acl bump_nobumpsites ssl::server_name "/usr/local/etc/squid/nobumpsites.acl"
2024/02/28 16:56:04| Processing: ssl_bump peek bump_step1 all
2024/02/28 16:56:04| Processing: ssl_bump peek bump_step2 bump_nobumpsites
2024/02/28 16:56:04| Processing: ssl_bump splice bump_step3 bump_nobumpsites
2024/02/28 16:56:04| Processing: ssl_bump stare bump_step2
2024/02/28 16:56:04| Processing: ssl_bump bump bump_step3
2024/02/28 16:56:04| Processing: sslproxy_cert_error deny all
2024/02/28 16:56:04| Processing: acl ftp proto FTP
2024/02/28 16:56:04| Processing: http_access allow ftp
2024/02/28 16:56:04| Processing: acl localnet src 10.100.0.0/16 # Possible internal network (interfaces v4)
2024/02/28 16:56:04| Processing: acl localnet src fc00::/7       # RFC 4193 local private network range
2024/02/28 16:56:04| Processing: acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines
2024/02/28 16:56:04| Processing: acl PURGE method PURGE
2024/02/28 16:56:04| Processing: http_access allow localhost PURGE
2024/02/28 16:56:04| Processing: http_access deny PURGE
2024/02/28 16:56:04| Processing: acl subnets src 10.100.0.0/16
2024/02/28 16:56:04| Processing: acl subnets src 192.168.0.0/24
2024/02/28 16:56:04| Processing: acl whiteList url_regex "*\.discord\.com"
2024/02/28 16:56:04| ERROR: Can not open file *\.discord\.com for reading
2024/02/28 16:56:04| WARNING: empty ACL: acl whiteList url_regex "*\.discord\.com"
2024/02/28 16:56:04| Processing: acl whiteList url_regex "*\.whatsapp\.com"
2024/02/28 16:56:04| ERROR: Can not open file *\.whatsapp\.com for reading
2024/02/28 16:56:04| Processing: acl whiteList url_regex "*\.whatsapp\.net"
2024/02/28 16:56:04| ERROR: Can not open file *\.whatsapp\.net for reading
2024/02/28 16:56:04| Processing: acl whiteList url_regex "*\.facebook\.com"
2024/02/28 16:56:04| ERROR: Can not open file *\.facebook\.com for reading
2024/02/28 16:56:04| Processing: acl SSL_ports port 443 # https
2024/02/28 16:56:04| Processing: acl Safe_ports port 80 # http
2024/02/28 16:56:04| Processing: acl Safe_ports port 21 # ftp
2024/02/28 16:56:04| Processing: acl Safe_ports port 443 # https
2024/02/28 16:56:04| Processing: acl Safe_ports port 70 # gopher
2024/02/28 16:56:04| Processing: acl Safe_ports port 210 # wais
2024/02/28 16:56:04| Processing: acl Safe_ports port 1025-65535 # unregistered ports
2024/02/28 16:56:04| Processing: acl Safe_ports port 280 # http-mgmt
2024/02/28 16:56:04| Processing: acl Safe_ports port 488 # gss-http
2024/02/28 16:56:04| Processing: acl Safe_ports port 591 # filemaker
2024/02/28 16:56:04| Processing: acl Safe_ports port 777 # multiling http
2024/02/28 16:56:04| Processing: acl CONNECT method CONNECT
2024/02/28 16:56:04| Processing: icap_enable on
2024/02/28 16:56:04| Processing: icap_default_options_ttl 60
2024/02/28 16:56:04| Processing: adaptation_send_client_ip on
2024/02/28 16:56:04| Processing: adaptation_send_username off
2024/02/28 16:56:04| Processing: icap_client_username_encode off
2024/02/28 16:56:04| Processing: icap_client_username_header X-Username
2024/02/28 16:56:04| Processing: icap_preview_enable on
2024/02/28 16:56:04| Processing: icap_preview_size 1024
2024/02/28 16:56:04| Processing: icap_service response_mod respmod_precache icap://[::1]:1344/avscan
2024/02/28 16:56:04| Processing: icap_service request_mod reqmod_precache icap://[::1]:1344/avscan
2024/02/28 16:56:04| Processing: include /usr/local/etc/squid/pre-auth/*.conf
2024/02/28 16:56:04| Processing Configuration File: /usr/local/etc/squid/pre-auth/40-snmp.conf (depth 1)
2024/02/28 16:56:04| Processing Configuration File: /usr/local/etc/squid/pre-auth/dummy.conf (depth 1)
2024/02/28 16:56:04| Processing Configuration File: /usr/local/etc/squid/pre-auth/intermediate-ca.conf (depth 1)
2024/02/28 16:56:04| Processing: acl fetched_certificate transaction_initiator certificate-fetching
2024/02/28 16:56:04| Processing: cache allow fetched_certificate
2024/02/28 16:56:04| Processing: http_access allow fetched_certificate
2024/02/28 16:56:04| Processing Configuration File: /usr/local/etc/squid/pre-auth/parentproxy.conf (depth 1)
2024/02/28 16:56:04| Processing: adaptation_access response_mod allow whiteList
2024/02/28 16:56:04| Processing: adaptation_access request_mod allow whiteList
2024/02/28 16:56:04| Processing: http_access allow whiteList
2024/02/28 16:56:04| Processing: adaptation_access response_mod deny !Safe_ports
2024/02/28 16:56:04| Processing: adaptation_access request_mod deny !Safe_ports
2024/02/28 16:56:04| Processing: http_access deny !Safe_ports
2024/02/28 16:56:04| Processing: adaptation_access response_mod deny CONNECT !SSL_ports
2024/02/28 16:56:04| Processing: adaptation_access request_mod deny CONNECT !SSL_ports
2024/02/28 16:56:04| Processing: http_access deny CONNECT !SSL_ports
2024/02/28 16:56:04| Processing: http_access allow localhost manager
2024/02/28 16:56:04| Processing: http_access deny manager
2024/02/28 16:56:04| Processing: adaptation_access response_mod deny to_localhost
2024/02/28 16:56:04| Processing: adaptation_access request_mod deny to_localhost
2024/02/28 16:56:04| Processing: http_access deny to_localhost
2024/02/28 16:56:04| Processing: include /usr/local/etc/squid/auth/*.conf
2024/02/28 16:56:04| Processing Configuration File: /usr/local/etc/squid/auth/dummy.conf (depth 1)
2024/02/28 16:56:04| Processing: adaptation_access response_mod allow localnet
2024/02/28 16:56:04| Processing: adaptation_access request_mod allow localnet
2024/02/28 16:56:04| Processing: http_access allow localnet
2024/02/28 16:56:04| Processing: adaptation_access response_mod allow localhost
2024/02/28 16:56:04| Processing: adaptation_access request_mod allow localhost
2024/02/28 16:56:04| Processing: http_access allow localhost
2024/02/28 16:56:04| Processing: adaptation_access response_mod allow subnets
2024/02/28 16:56:04| Processing: adaptation_access request_mod allow subnets
2024/02/28 16:56:04| Processing: http_access allow subnets
2024/02/28 16:56:04| Processing: adaptation_access response_mod deny all
2024/02/28 16:56:04| Processing: adaptation_access request_mod deny all
2024/02/28 16:56:04| Processing: http_access deny all
2024/02/28 16:56:04| Processing: include /usr/local/etc/squid/post-auth/*.conf
2024/02/28 16:56:04| Processing Configuration File: /usr/local/etc/squid/post-auth/dummy.conf (depth 1)
2024/02/28 16:56:04| Processing Configuration File: /usr/local/etc/squid/post-auth/rewrite_allow.conf (depth 1)
2024/02/28 16:56:04| Processing: url_rewrite_access deny CONNECT
2024/02/28 16:56:04| Processing: url_rewrite_access allow all
2024/02/28 16:56:04| Processing Configuration File: /usr/local/etc/squid/post-auth/tls.conf (depth 1)
2024/02/28 16:56:04| Processing: tls_outgoing_options min-version=1.1 cipher=HIGH:MEDIUM:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS
2024/02/28 16:56:04| Processing: sslproxy_foreign_intermediate_certs /usr/local/etc/squid/extra-intermediate-CA.pem
2024/02/28 16:56:04| Processing: cache_mem 1024 MB
2024/02/28 16:56:04| Processing: cache_dir ufs /var/squid/cache 512 16 256
2024/02/28 16:56:04| Processing: coredump_dir /var/squid/cache
2024/02/28 16:56:04| Processing: refresh_pattern ^ftp:          1440    20%     10080
2024/02/28 16:56:04| Processing: refresh_pattern ^gopher:       1440    0%      1440
2024/02/28 16:56:04| Processing: refresh_pattern -i (/cgi-bin/|\?) 0    0%      0
2024/02/28 16:56:04| Processing: refresh_pattern .              0       20%     4320
2024/02/28 16:56:04| Processing: access_log stdio:/var/log/squid/access.log squid
2024/02/28 16:56:04| Processing: cache_store_log stdio:/var/log/squid/store.log
2024/02/28 16:56:04| Processing: httpd_suppress_version_string on
2024/02/28 16:56:04| Processing: uri_whitespace strip
2024/02/28 16:56:04| Processing: forwarded_for truncate
2024/02/28 16:56:04| Processing: logfile_rotate 0
2024/02/28 16:56:04| Processing: visible_hostname webproxy.redacted
2024/02/28 16:56:04| Processing: cache_mgr hostmaster@redacted
2024/02/28 16:56:04| Processing: error_directory /usr/local/etc/squid/errors/local
2024/02/28 16:56:04| Requiring client certificates.
2024/02/28 16:56:04| Loaded signing certificate: /C=AR/ST=Buenos Aires/L=Capital Federal/O=Redacted/emailAddress=hostmaster@redacted/CN=redacted
2024/02/28 16:56:04| Not requiring any client certificates
Segmentation fault (core dumped)

Edit: Here's the final few lines of output with full debug verbosity enabled:

2024/02/28 17:03:08| Processing: error_directory /usr/local/etc/squid/errors/local
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf47
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf47 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf47 no cow needed; have 35
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf116
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf116 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf116 no cow needed; have 35
2024/02/28 17:03:08.911| 28,3| InnerNode.cc(58) lineParse: looking for ACL all
2024/02/28 17:03:08.911| 28,9| Acl.cc(121) FindByName: ACL::FindByName 'all'
2024/02/28 17:03:08.911| 45,9| cbdata.cc(168) cbdataInternalAlloc: Allocating 0x82e0659d8
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf47
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf47 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf47 no cow needed; have 35
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf116
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf116 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf116 no cow needed; have 35
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf47
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf47 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf47 no cow needed; have 35
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf116
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf116 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf116 no cow needed; have 35
2024/02/28 17:03:08.911| 80,5| wccp2.cc(513) wccp2_add_service_list: wccp2_add_service_list: added service id 0
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf47
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf47 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf47 no cow needed; have 35
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf116
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf116 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf116 no cow needed; have 35
2024/02/28 17:03:08.911| 28,3| InnerNode.cc(58) lineParse: looking for ACL all
2024/02/28 17:03:08.911| 28,9| Acl.cc(121) FindByName: ACL::FindByName 'all'
2024/02/28 17:03:08.911| 45,9| cbdata.cc(168) cbdataInternalAlloc: Allocating 0x82e065a98
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf47
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf47 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf47 no cow needed; have 35
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf116
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf116 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf116 no cow needed; have 35
2024/02/28 17:03:08.911| 28,3| InnerNode.cc(58) lineParse: looking for ACL ssl::certUntrusted
2024/02/28 17:03:08.911| 28,9| Acl.cc(121) FindByName: ACL::FindByName 'ssl::certUntrusted'
2024/02/28 17:03:08.911| 45,9| cbdata.cc(168) cbdataInternalAlloc: Allocating 0x82e065b58
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf47
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf47 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf47 no cow needed; have 35
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf116
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf116 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf116 no cow needed; have 35
2024/02/28 17:03:08.911| 28,3| InnerNode.cc(58) lineParse: looking for ACL ssl::certSelfSigned
2024/02/28 17:03:08.911| 28,9| Acl.cc(121) FindByName: ACL::FindByName 'ssl::certSelfSigned'
2024/02/28 17:03:08.911| 45,9| cbdata.cc(168) cbdataInternalAlloc: Allocating 0x82e065c18
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf47
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf47 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf47 no cow needed; have 35
2024/02/28 17:03:08.911| 24,7| SBuf.cc(160) rawSpace: reserving 1 for SBuf116
2024/02/28 17:03:08.911| 24,8| SBuf.cc(880) cow: SBuf116 new size:6
2024/02/28 17:03:08.911| 24,7| MemBlob.cc(130) syncSize: 5 was: 6
2024/02/28 17:03:08.911| 24,8| SBuf.cc(891) cow: SBuf116 no cow needed; have 35
2024/02/28 17:03:08.911| 28,3| InnerNode.cc(58) lineParse: looking for ACL all
2024/02/28 17:03:08.911| 28,9| Acl.cc(121) FindByName: ACL::FindByName 'all'
2024/02/28 17:03:08.911| 45,9| cbdata.cc(168) cbdataInternalAlloc: Allocating 0x82e065cd8
Segmentation fault (core dumped)

I can upload the core dump file if need-be.

dblanque avatar Feb 28 '24 19:02 dblanque

Update: Seems to be an issue with reloading/parsing the config file or using the squid binary whenever the service is already running. Stopping and starting the service manually seems to work fine.

dblanque avatar Feb 28 '24 21:02 dblanque

I can confirm that this issue still happens on 24.1.2_1.

Trying the workaroung from @dblanque brings up the following error message in the GUI but the service is being started in the background:

Segmentation fault
Starting squid.
Segmentation fault (core dumped)
/usr/local/etc/rc.d/squid: WARNING: failed to start squid

So don't be confused by the messages and always check the service status manually.

codiflow avatar Mar 02 '24 11:03 codiflow

could be https://github.com/squid-cache/squid/commit/0129396e16573b92e7e0c81c46653f5d488968e6 , which might point to a logger issue in the configuration.

AdSchellevis avatar Mar 02 '24 11:03 AdSchellevis

I've uploaded new snapshots with the patch in place, so again:

# opnsense-revert -z squid

Restart isn't needed if it's "just" squid -k crashing, which is more or less a cosmetic problem

Cheers, Franco

fichtner avatar Mar 04 '24 14:03 fichtner

Will test it today on a non-operative time-frame and get back to you on the result of the hotfix asap. Thank you Franco.

dblanque avatar Mar 04 '24 14:03 dblanque

Hey Franco, Sadly that did not fix the issue.

If there is any more info I can get for you regarding this, let me know. Regards, Dylan

dblanque avatar Mar 05 '24 00:03 dblanque

Good day. I encountered exactly the same problem. None of the above helps.

Dervud avatar Mar 05 '24 02:03 Dervud

The seg fault comes from /usr/local/opnsense/scripts/proxy/setup.sh. Here is output with set -x from that:

root@edge:/etc/rc.d # service squid reload
+ SQUID_DIRS='/var/log/squid /var/run/squid /var/squid /var/squid/cache /var/squid/ssl /var/squid/logs /usr/local/etc/squid/errors/local'
+ mkdir -p /var/log/squid
+ chown -R squid:squid /var/log/squid
+ chmod -R 750 /var/log/squid
+ mkdir -p /var/run/squid
+ chown -R squid:squid /var/run/squid
+ chmod -R 750 /var/run/squid
+ mkdir -p /var/squid
+ chown -R squid:squid /var/squid
+ chmod -R 750 /var/squid
+ mkdir -p /var/squid/cache
+ chown -R squid:squid /var/squid/cache
+ chmod -R 750 /var/squid/cache
+ mkdir -p /var/squid/ssl
+ chown -R squid:squid /var/squid/ssl
+ chmod -R 750 /var/squid/ssl
+ mkdir -p /var/squid/logs
+ chown -R squid:squid /var/squid/logs
+ chmod -R 750 /var/squid/logs
+ mkdir -p /usr/local/etc/squid/errors/local
+ chown -R squid:squid /usr/local/etc/squid/errors/local
+ chmod -R 750 /usr/local/etc/squid/errors/local
+ /usr/sbin/pw groupmod proxy -m squid
+ /usr/local/sbin/squid -z -N
Segmentation fault
+ [ -f /usr/local/etc/squid/ca.pem.id ]
+ cat /usr/local/etc/squid/ca.pem.id
+ current_cert=6141ee159c02a
+ [ -d /var/squid/ssl_crtd ]
+ [ -f /var/squid/ssl_crtd.id ]
+ cat /var/squid/ssl_crtd.id
+ running_cert=6141ee159c02a
+ [ 6141ee159c02a '!=' 6141ee159c02a ]
+ [ ! -d /var/squid/ssl_crtd ]
+ /usr/local/opnsense/scripts/proxy/generate_cert.php
+ /usr/local/opnsense/scripts/proxy/deploy_error_pages.py
Performing sanity check on squid configuration.
2024/03/06 19:47:24| Processing Configuration File: /usr/local/etc/squid/squid.conf (depth 0)
2024/03/06 19:47:24| Processing Configuration File: /usr/local/etc/squid/pre-auth/40-snmp.conf (depth 1)
2024/03/06 19:47:24| Processing Configuration File: /usr/local/etc/squid/pre-auth/dummy.conf (depth 1)
2024/03/06 19:47:24| Processing Configuration File: /usr/local/etc/squid/pre-auth/parentproxy.conf (depth 1)
2024/03/06 19:47:24| Processing Configuration File: /usr/local/etc/squid/auth/dummy.conf (depth 1)
2024/03/06 19:47:24| Processing Configuration File: /usr/local/etc/squid/post-auth/dummy.conf (depth 1)
2024/03/06 19:47:24| Set Current Directory to /var/squid/cache
Segmentation fault
root@edge:/etc/rc.d #

grimm26 avatar Mar 07 '24 01:03 grimm26

Update: Seems to be an issue with reloading/parsing the config file or using the squid binary whenever the service is already running. Stopping and starting the service manually seems to work fine.

That does not seem to be the case. I can fully stop squid and do something simple like squid -k check and get a seg fault. It drops a squid.core in /var/squid so I ran a strings on that and I do see a

Invalid shared object handle 0x82ad391b0
__vdso_ia32_sigcode
__vdso_freebsd4_ia32_sigcode
elf-vdso32.so.1

towards the end of it. Not sure if those lines being together means anything. I was going to compile squid myself and see if that also threw an error, but I'm not a FreeBSD guy and the make failed.

grimm26 avatar Mar 07 '24 21:03 grimm26

I just uploaded a squid build for 6.8. setup.sh or not if squid binary crashes we can't do that much I'm afraid. But I'm repeating myself.

To install the 6.8 version:

# opnsense-revert -z squid

Best to test with a clean reboot afterwards.

Cheers, Franco

fichtner avatar Mar 08 '24 10:03 fichtner

@fichtner squid 6.8 still segfaults for me like 6.7 did.

grimm26 avatar Mar 08 '24 14:03 grimm26

Fair enough. Was worth a try. ;)

fichtner avatar Mar 08 '24 15:03 fichtner

I've had the exact same issue, so I was trying a lot of things, and I think one of these two things fixed it: A) In the general settings, I changed the error page dropdown from OPNsense -> Squid B) I then unchcked the 'Proxy enabled' and applied, and then rechecked and applied, and it started correctly. This was after trying to restart multiple times, including from command line, all giving segfaults. I'm not sure which of these two was actually the key, but maybe this points someone else in the right direction.

gridstop avatar Mar 12 '24 02:03 gridstop

@gridstop I did that and it did not help. Still get a seg fault.

grimm26 avatar Mar 12 '24 15:03 grimm26

I've had the exact same issue, so I was trying a lot of things, and I think one of these two things fixed it: A) In the general settings, I changed the error page dropdown from OPNsense -> Squid B) I then unchcked the 'Proxy enabled' and applied, and then rechecked and applied, and it started correctly. This was after trying to restart multiple times, including from command line, all giving segfaults. I'm not sure which of these two was actually the key, but maybe this points someone else in the right direction.

it didn't work either :(

pablopol24 avatar Mar 12 '24 18:03 pablopol24

Well, sorry about that and nevermind then. I looked through the config again, I genuinely can't see anything else I changed, and it just suddenly started working when I unchecked & rechecked the enable box. It had never started successfully since upgrading to 24.1, including not starting on reboots.

gridstop avatar Mar 12 '24 19:03 gridstop

suffer from Segmentation fault how to resolve

root@OPNsense:~ # service squid reload Segmentation fault Performing sanity check on squid configuration. 2024/03/13 16:47:42| Processing Configuration File: /usr/local/etc/squid/squid.conf (depth 0) 2024/03/13 16:47:42| Processing Configuration File: /usr/local/etc/squid/pre-auth/40-snmp.conf (depth 1) 2024/03/13 16:47:42| Processing Configuration File: /usr/local/etc/squid/pre-auth/dummy.conf (depth 1) 2024/03/13 16:47:42| Processing Configuration File: /usr/local/etc/squid/pre-auth/parentproxy.conf (depth 1) 2024/03/13 16:47:42| Processing Configuration File: /usr/local/etc/squid/auth/dummy.conf (depth 1) 2024/03/13 16:47:42| Processing Configuration File: /usr/local/etc/squid/post-auth/dummy.conf (depth 1) 2024/03/13 16:47:42| Set Current Directory to /var/squid/cache Segmentation fault

Then i have applied opnsense-revert -z squid still getting same error

devmalick avatar Mar 13 '24 11:03 devmalick

I have affected installations too. Nothing fancy. Just os-squid and even removed anything left of older squid installations from the system and from config.xml. Still I observe this:

root:/usr/local/etc/squid # squid -d9 -f squid.conf -k parse                                                                                                                                               2024/03/18 12:04:29| Processing Configuration File: squid.conf (depth 0)
2024/03/18 12:04:29| Processing: http_port 10.0.0.100:3128
2024/03/18 12:04:29| Processing: acl ftp proto FTP
2024/03/18 12:04:29| Processing: http_access allow ftp
2024/03/18 12:04:29| Processing: acl localnet src 10.63.36.0/23 # Possible internal network (interfaces v4)
2024/03/18 12:04:29| Processing: acl localnet src fc00::/7       # RFC 4193 local private network range
2024/03/18 12:04:29| Processing: acl localnet src fe80::/10      # RFC 4291 link-local (directly plugged) machines
2024/03/18 12:04:29| Processing: acl unrestricted src 10.0.0.1
2024/03/18 12:04:29| Processing: acl unrestricted src 10.0.0.2
2024/03/18 12:04:29| Processing: acl unrestricted src 10.0.0.3
2024/03/18 12:04:29| Processing: acl unrestricted src 10.0.0.4
2024/03/18 12:04:29| Processing: acl SSL_ports port 443 # https
2024/03/18 12:04:29| Processing: acl Safe_ports port 80 # http
2024/03/18 12:04:29| Processing: acl Safe_ports port 443 # https
2024/03/18 12:04:29| Processing: acl CONNECT method CONNECT
2024/03/18 12:04:29| Processing: icap_enable off
2024/03/18 12:04:29| Processing: include /usr/local/etc/squid/pre-auth/*.conf
2024/03/18 12:04:29| Processing Configuration File: /usr/local/etc/squid/pre-auth/40-snmp.conf (depth 1)
2024/03/18 12:04:29| Processing Configuration File: /usr/local/etc/squid/pre-auth/dummy.conf (depth 1)
2024/03/18 12:04:29| Processing Configuration File: /usr/local/etc/squid/pre-auth/parentproxy.conf (depth 1)
2024/03/18 12:04:29| Processing: http_access allow unrestricted
2024/03/18 12:04:29| Processing: http_access deny !Safe_ports !unrestricted
2024/03/18 12:04:29| Processing: http_access deny CONNECT !SSL_ports !unrestricted
2024/03/18 12:04:29| Processing: http_access allow localhost manager
2024/03/18 12:04:29| Processing: http_access deny manager
2024/03/18 12:04:29| Processing: http_access deny to_localhost
2024/03/18 12:04:29| Processing: include /usr/local/etc/squid/auth/*.conf
2024/03/18 12:04:29| Processing Configuration File: /usr/local/etc/squid/auth/dummy.conf (depth 1)
2024/03/18 12:04:29| Processing: http_access allow localnet
2024/03/18 12:04:29| Processing: http_access allow localhost
2024/03/18 12:04:29| Processing: http_access deny all
2024/03/18 12:04:29| Processing: include /usr/local/etc/squid/post-auth/*.conf
2024/03/18 12:04:29| Processing Configuration File: /usr/local/etc/squid/post-auth/dummy.conf (depth 1)
2024/03/18 12:04:29| Processing: cache_mem 256 MB
2024/03/18 12:04:29| Processing: coredump_dir /var/squid/cache
2024/03/18 12:04:29| Processing: refresh_pattern ^ftp:          1440    20%     10080
2024/03/18 12:04:29| Processing: refresh_pattern ^gopher:       1440    0%      1440
2024/03/18 12:04:29| Processing: refresh_pattern -i (/cgi-bin/|\?) 0    0%      0
2024/03/18 12:04:29| Processing: refresh_pattern .              0       20%     4320
2024/03/18 12:04:29| Processing: pinger_enable off
2024/03/18 12:04:29| Processing: access_log stdio:/var/log/squid/access.log squid
2024/03/18 12:04:29| Processing: cache_store_log stdio:/var/log/squid/store.log
2024/03/18 12:04:29| Processing: logfile_rotate 0
2024/03/18 12:04:29| Processing: cache_mgr admins
2024/03/18 12:04:29| Processing: error_directory /usr/local/share/squid-langpack/en
2024/03/18 12:04:29| Requiring client certificates.
Segmentation fault
root:/usr/local/etc/squid #

Edit: to be a bit more clear and avoid mistakes. When I say "nothing fancy" and "plain squid", this is still a system where older versions of OPNSense were installed. I ran through the system manually and tried to cleanup everything. So it was not a "clean install" of OPNSense.

wiigoac avatar Mar 18 '24 11:03 wiigoac

With a little bit more playing around. I would say it has to be more the "checking of config" part than the actual "running proxy" part of the story. As was mentioned before. You can get this:

image

and have a running squid afterwards. Also I can run the process manual in foreground and seems to be ok, while segfaulting on the "-k parse". I would love to give more secific information, but I am afraid I'll not have too much time to invest. Although, if there is something to try out, I will do so, of course.

wiigoac avatar Mar 18 '24 11:03 wiigoac

it also happens on mine, even with # opnsense-revert -z squid 6.8

ssteeltm avatar Mar 26 '24 14:03 ssteeltm

Oof, since 24.1.4 we've been getting intermittent crashes on squid. There's something crooked with Squid on the new version.

dblanque avatar Mar 26 '24 14:03 dblanque

+1 i have the:

Segmentation fault
Performing sanity check on squid configuration.
2024/03/21 17:38:22| Processing Configuration File: /usr/local/etc/squid/squid.conf (depth 0)
2024/03/21 17:38:22| Starting Authentication on port 127.0.0.1:3128
2024/03/21 17:38:22| Disabling Authentication on port 127.0.0.1:3128 (interception enabled)
2024/03/21 17:38:22| Starting Authentication on port [::1]:3128
2024/03/21 17:38:22| Disabling Authentication on port [::1]:3128 (interception enabled)
2024/03/21 17:38:22| Starting Authentication on port 127.0.0.1:3129
2024/03/21 17:38:22| Disabling Authentication on port 127.0.0.1:3129 (interception enabled)
2024/03/21 17:38:22| Starting Authentication on port [::1]:3129
2024/03/21 17:38:22| Disabling Authentication on port [::1]:3129 (interception enabled)
2024/03/21 17:38:22| Processing Configuration File: /usr/local/etc/squid/pre-auth/40-snmp.conf (depth 1)
2024/03/21 17:38:22| Processing Configuration File: /usr/local/etc/squid/pre-auth/dummy.conf (depth 1)
2024/03/21 17:38:22| Processing Configuration File: /usr/local/etc/squid/pre-auth/parentproxy.conf (depth 1)
2024/03/21 17:38:22| Processing Configuration File: /usr/local/etc/squid/auth/dummy.conf (depth 1)
2024/03/21 17:38:22| Processing Configuration File: /usr/local/etc/squid/post-auth/dummy.conf (depth 1)
2024/03/21 17:38:22| WARNING: HTTP requires the use of Via
2024/03/21 17:38:23| Set Current Directory to /var/squid/cache
Segmentation fault

on service restart and

Segmentation fault
Starting squid.
Segmentation fault (core dumped)
/usr/local/etc/rc.d/squid: WARNING: failed to start squid

on service start

in addition, i need to wait 5 minutes after Opnsense started before squid starts to work properly. before that i will get "no error" page from squid if try to reach any web site

PS: a had reinstalled clean and reconfigure Opnsense by hand (not loading settings) on new VM (empty volume) in struggle to get rid of the issue. unsuccessfully.

densu90 avatar Mar 26 '24 19:03 densu90

To exclude configuration issues in our templates, I tried to execute squid using the /usr/local/etc/squid/squid.conf.sample file as well, which also seems to coredump. At a first glance it dies on initializing the client tls context. When disabling tls_outgoing_options, squid -k parse exists without coredump.

tls_outgoing_options disable

When increasing debug output, I instantly hit the next coredump.... which also makes it impossible to trace errors inside the tls_outgoing_options block as it will never reach there with -X appended (on my end)

AdSchellevis avatar Mar 27 '24 19:03 AdSchellevis