testssl.sh
testssl.sh copied to clipboard
drwetter
Hangs before testing begins
Please make sure that you provide enough information so that we understand what your issue is about.
- Did you check the documentation in ~/doc/ or, if it is a different problem: Did you google for it?
Yep - similar to #1489 but now much worse.
- uname -a
Linux DESKTOP-G6CKCF5 4.4.0-19041-Microsoft #488-Microsoft Mon Sep 01 13:43:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux
- testssl version from the banner: testssl.sh -b 2>/dev/null | head -4 | tail -2
The above command hangs.
- git log | head -1 (if running from git repo)
I'm running the 3.0.2 zip downloaded into an Ubuntu 20.04 instance running under WSL2 on Win10. The same behaviour occurs from the latest git which I tried initially.
- openssl version used by testssl.sh: testssl.sh -b 2>/dev/null | awk -F':' '/openssl/ { print $2}'
Hangs.
- steps to reproduce: testssl.sh or docker command line, if possible incl. host
Via docker it seems to work fine.
- what exactly was happening, output is needed
➜ testssl.sh-3.0.2 ./testssl.sh https://www.google.com
No engine or GOST support via engine with your /usr/bin/openssl
^C
Waited several minutes before hitting ctrl+c.
- what did you expect instead?
A test...
-v pls. At least the hanging process needs to be identified. You may want to have a look @ 'https://github.com/drwetter/testssl.sh/wiki/Findings-and-HowTo-Fix-them (Debug yourself).
Cheers, Dirk
As requested.
➜ testssl.sh-3.0.2 time ./testssl.sh -v https://www.bbc.co.uk
No engine or GOST support via engine with your /usr/bin/openssl
###########################################################
testssl.sh 3.0.2 from https://testssl.sh/
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~79 ciphers]
on DESKTOP-G6CKCF5:/usr/bin/openssl
(built: "Apr 20 11:53:50 2020", platform: "debian-amd64")
./testssl.sh -v https://www.bbc.co.uk 0.23s user 2.36s system 0% cpu 12:18.91 total
I'll take a look at the document but this thing always used to "just work" under an older environment. Not sure if it's that I'm now using WSL v2 or an updated Ubuntu that is causing the issue.
by -v I was like asking for more verbose input from you, James, not from testssl.sh ;-) (sorry)
I'm not sure how I can be of much help. Your document says to run the whole script. I'm guessing that you expect people to be including your tool in their own script? Can't see any other references to a script.
I ran with --debug=2 --log and I have to say it all looks pretty normal once it becomes unstuck...
## Scan started as: "testssl.sh --debug=2 --log https://www.bbc.co.uk"
## at DESKTOP-G6CKCF5:/usr/bin/openssl
## version testssl: 3.0.2 from
## version openssl: "1.1.1f" from "Apr 20 11:53:50 2020")
Testing all IPv4 addresses (port 443): 212.58.237.252 212.58.233.252
------------------------------------------------------------------------------------------
Start 2020-10-01 10:45:09 -->> 212.58.237.252:443 (www.bbc.co.uk) <<--
Further IP addresses: 212.58.233.252
rDNS (212.58.237.252): --
sending client hello... sending client hello... reading server hello...
sending close_notify...
(286 lines returned)
sending client hello... sending client hello... reading server hello...
sending close_notify...
(276 lines returned)
one proto determined: tls1_3
OPTIMAL_PROTO:
HTTP/1.1 200 OK
Date: Thu, 01 Oct 2020 09:45:12 GMT
...
I've just re-run with --debug=6. It immediately spits out k=v options, warns about GOST support, then hangs. Output appears identical to above.
The following records exactly what happens. Make yourself a coffee in the middle! https://asciinema.org/a/362904
Hi James,
can't still tell where it hangs. You need me pls either to provide the command in the process list (ps fawux) or, better: SETX=true bash -x testssl.sh <CMDLINE>". When you run the latter you'll spot the culprit
asciicinema is great. I always wanted to amend the description, see #1242 . Maybe with a little bit more of action than yours ;-)
Cheers, Dirk
Hope this helps then:
|16952> find_openssl_binary(): HAS_CHACHA20=false
|16953> find_openssl_binary(): HAS_AES128_GCM=false
|16954> find_openssl_binary(): HAS_AES256_GCM=false
|16955> find_openssl_binary(): HAS_ZLIB=false
|16957> find_openssl_binary(): /usr/bin/openssl ciphers -s
|16957> find_openssl_binary(): grep -aiq 'unknown option'
|16958> find_openssl_binary(): OSSL_CIPHERS_S=-s
|16962> find_openssl_binary(): /usr/bin/openssl s_client -ssl2 -connect invalid.
|16962> find_openssl_binary(): grep -aiq 'unknown option'
|16965> find_openssl_binary(): /usr/bin/openssl s_client -ssl3 -connect invalid.
|16965> find_openssl_binary(): grep -aiq 'unknown option'
|16968> find_openssl_binary(): /usr/bin/openssl s_client -tls1_3 -connect invalid.
|16968> find_openssl_binary(): grep -aiq 'unknown option'
^
ctrl+c obviously applied.
Thanks. Strange though
What does
/usr/bin/openssl version -areturn/usr/bin/openssl s_client -tls1_3 -connect invalid.return (mind the trailing dot here)/usr/bin/openssl genpkey -algorithm X448return
➜ testssl.sh-3.0.2 /usr/bin/openssl version -a
OpenSSL 1.1.1f 31 Mar 2020
built on: Mon Apr 20 11:53:50 2020 UTC
platform: debian-amd64
options: bn(64,64) rc4(16x,int) des(int) blowfish(ptr)
compiler: gcc -fPIC -pthread -m64 -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-P_ODHM/openssl-1.1.1f=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_TLS_SECURITY_LEVEL=2 -DOPENSSL_USE_NODELETE -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DRC4_ASM -DMD5_ASM -DAESNI_ASM -DVPAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DX25519_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
OPENSSLDIR: "/usr/lib/ssl"
ENGINESDIR: "/usr/lib/x86_64-linux-gnu/engines-1.1"
Seeding source: os-specific
➜ testssl.sh-3.0.2 /usr/bin/openssl s_client -tls1_3 -connect invalid.
connect:errno=11
(the above hung for a while)
➜ testssl.sh-3.0.2 /usr/bin/openssl genpkey -algorithm X448
-----BEGIN PRIVATE KEY-----
MEYCAQAwBQYDK2VvBDoEOAyB0AK7epn2ReazViZck+R4b9yFjsKB/WQ87ABoXWqb
kYcs2JBD5Rg/ZaVMqalPXCq6AxMZvJbN
-----END PRIVATE KEY-----
Not that it is of any help for you but your /usr/bin/openssl seems not to work in your context (WSL).
Does /usr/bin/openssl s_client -tls1_3 -connect testssl.sh:443 -servername testssl.sh </dev/null work?
And: Does is still hang when you swap invalid. with test. example. localhost. or x (no trailing dot for x) ?
The only workarounds which I can imagine right now are not nice ones.
/usr/bin/openssl s_client -tls1_3 -connect testssl.sh:443 -servername testssl.sh </dev/null comes back just fine.
With test. - no, with example. - no, with localhost. - immediate connection refused messages, with x - no.
sigh.
I guess it's a DNS issue when /usr/bin/openssl s_client -tls1_2 -connect invalid. doesn't work either?
➜ testssl.sh-3.0.2 /usr/bin/openssl s_client -tls1_2 -connect invalid.
connect:errno=11
The rest of the environment does seem to work. It's reason for being is git, and other linux based tooling like aws cli.
invalid., test. and example. are legitimate names (https://tools.ietf.org/html/rfc6761) and your DNS resolver should return NXDOMAIN.
There's something broken with your DNS config or with WSL on your side I can't help you with. Sorry
To be clear, are you suggesting the tool requires those to operate?
Sarcastically: Yes, the tool requires to have a proper DNS resolver. This is propaby not what you want to hear but what do you expect me to do without breaking other setups?
We need to make a check whether TLS 1.3 is natively supported. We spent a lot of time getting this check to work -- for probably everybody except your setup. Or maybe for Microsoft's broken implementation.
In your setup the first I would recommend to do is to understand why this fails. Maybe it's a config problem, maybe it's an intrinsic issue of WSL2. If you don't want that which I can understand, you need to either to change the platform or patch privately the line to HAS_TLS13=true or maybe a local DNS entry for invalid. or invalid works (Windows' /etc/hosts or WSL's?)
Hi @jmkgreen ,
could you please try
prompt> for t in invalid. test. example. test.; do
time /usr/bin/openssl s_client -tls1_2 -connect $t
done
prompt>
Idea is to make the Special-Use Domain Names configurable. It seems a more common problem as I assumed a while back.
I'm experiencing similar slowness in a WSL2 environment. I've run your latest command above, and get the following:
for t in invalid. test. example. test.; do
> time /usr/bin/openssl s_client -tls1_2 -connect $t
> done
140264340821312:error:2008F002:BIO routines:BIO_lookup_ex:system lib:../crypto/bio/b_addr.c:726:Name or service not known
connect:errno=22
real 0m7.222s
user 0m0.006s
sys 0m0.000s
140247974712640:error:2008F002:BIO routines:BIO_lookup_ex:system lib:../crypto/bio/b_addr.c:726:Temporary failure in name resolution
connect:errno=11
real 0m20.023s
user 0m0.010s
sys 0m0.001s
139774726587712:error:2008F002:BIO routines:BIO_lookup_ex:system lib:../crypto/bio/b_addr.c:726:Name or service not known
connect:errno=22
real 0m15.017s
user 0m0.008s
sys 0m0.001s
140431012365632:error:2008F002:BIO routines:BIO_lookup_ex:system lib:../crypto/bio/b_addr.c:726:Name or service not known
connect:errno=22
real 0m2.049s
user 0m0.006s
sys 0m0.001s
Ok, least negative entries are cached - somehow.
Is there a non-existing windows domain name which resolves instantly?
Not sure! But here's something interesting. The same command runs much more quickly in Git Bash (stand-alone non-WSL linux environment which comes with git for Windows):
for t in invalid. test. example. test.; do
> time /usr/bin/openssl s_client -tls1_2 -connect $t
> done
34359738384:error:2008F002:BIO routines:BIO_lookup_ex:system lib:crypto/bio/b_addr.c:730:Name or service not known
connect:errno=88
real 0m5.086s
user 0m0.062s
sys 0m0.093s
34359738384:error:2008F002:BIO routines:BIO_lookup_ex:system lib:crypto/bio/b_addr.c:730:Name or service not known
connect:errno=88
real 0m2.428s
user 0m0.062s
sys 0m0.062s
34359738384:error:2008F002:BIO routines:BIO_lookup_ex:system lib:crypto/bio/b_addr.c:730:Name or service not known
connect:errno=88
real 0m3.495s
user 0m0.062s
sys 0m0.062s
34359738384:error:2008F002:BIO routines:BIO_lookup_ex:system lib:crypto/bio/b_addr.c:730:Name or service not known
connect:errno=88
real 0m2.474s
user 0m0.061s
sys 0m0.046s
Here is nslookup for the same domains in WSL:
for t in invalid. test. example. test.; do
time nslookup $t
done
Server: 172.17.232.241
Address: 172.17.232.241#53
** server can't find invalid: NXDOMAIN
real 0m2.511s
user 0m0.183s
sys 0m0.202s
Server: 172.17.232.241
Address: 172.17.232.241#53
** server can't find test: NXDOMAIN
real 0m1.075s
user 0m0.000s
sys 0m0.049s
Server: 172.17.232.241
Address: 172.17.232.241#53
** server can't find example: NXDOMAIN
real 0m10.076s
user 0m0.000s
sys 0m0.055s
Server: 172.17.232.241
Address: 172.17.232.241#53
** server can't find test: NXDOMAIN
real 0m12.244s
user 0m0.010s
sys 0m0.031s
Here's the same thing in PowerShell:
foreach ($t in @('invalid.', 'test.', 'example.', 'test.')) {
>> $time = Measure-Command { nslookup $t | Out-Default }
>> Write-Host $t completed in $time.TotalSeconds seconds`n
>> }
*** UnKnown can't find invalid.: Non-existent domain
Server: UnKnown
Address: fe80::1213:31ff:fe1b:952a
DNS request timed out.
timeout was 2 seconds.
invalid. completed in 2.1815786 seconds
*** UnKnown can't find test.: Non-existent domain
Server: UnKnown
Address: fe80::1213:31ff:fe1b:952a
test. completed in 0.3177416 seconds
*** UnKnown can't find example.: Non-existent domain
Server: UnKnown
Address: fe80::1213:31ff:fe1b:952a
example. completed in 0.1592713 seconds
*** UnKnown can't find test.: Non-existent domain
Server: UnKnown
Address: fe80::1213:31ff:fe1b:952a
test. completed in 0.1291524 seconds
Hi, After so many months with this issue, I finally found out this solution ! I don't know if it's the best solution but testssl starts now after 3 seconds instead of hanging forever.
Can you try the branch windows_dns_fix using NXCONNECT=localhost:0 ./testssl.sh $YOURTARGET and report back pls?
TL;DR
Yes! NXCONNECT=localhost:0 makes a dramatic difference in performance. Thanks!
Below are the outputs of the test domains above, both without and with NXCONNECT defined.
I have also tested to make sure that the speedup with NXCONNECT defined was not due to caching by rerunning the non-NXCONNECT script again, and I can confirm that the speedup is definitely due to defining NXCONNECT, not due to caching.
WITHOUT NXCONNECT
$ for t in invalid. test. example. test.; do
> time ./testssl.sh $t
> done
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Fatal error: No IPv4/IPv6 address(es) for "invalid" available
real 7m16.018s
user 0m2.873s
sys 0m1.822s
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Fatal error: No IPv4/IPv6 address(es) for "test" available
real 7m17.799s
user 0m3.434s
sys 0m2.094s
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Fatal error: No IPv4/IPv6 address(es) for "example" available
real 7m16.845s
user 0m3.301s
sys 0m1.829s
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Fatal error: No IPv4/IPv6 address(es) for "test" available
real 7m21.978s
user 0m3.721s
sys 0m2.875s
WITH NXCONNECT
$ for t in invalid. test. example. test.; do time NXCONNECT=localhost:0 ./testssl.sh $t; done
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Fatal error: No IPv4/IPv6 address(es) for "invalid" available
real 1m18.920s
user 0m3.297s
sys 0m2.357s
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Fatal error: No IPv4/IPv6 address(es) for "test" available
real 1m17.626s
user 0m3.566s
sys 0m2.038s
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Fatal error: No IPv4/IPv6 address(es) for "example" available
real 1m18.689s
user 0m3.772s
sys 0m2.421s
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Fatal error: No IPv4/IPv6 address(es) for "test" available
real 1m17.160s
user 0m3.473s
sys 0m2.152s
Hi @davidwales ,
slight misunderstanding. Just a NXCONNECT=localhost:0 -p ./testssl.sh $anyrealtarget maybe against testssl.sh -p $anyrealtarget would suffice (amended with -p).
Cheers, Dirk
So... This was without NXCONNECT:
$ time ./testssl.sh -p duckduckgo.com
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Start 2021-09-17 09:36:55 -->> 20.43.111.112:443 (duckduckgo.com) <<--
rDNS (20.43.111.112): --
Service detected: HTTP
Testing protocols via sockets except NPN+ALPN
SSLv2 not offered (OK)
SSLv3 not offered (OK)
TLS 1 not offered
TLS 1.1 not offered
TLS 1.2 offered (OK)
TLS 1.3 offered (OK): final
NPN/SPDY not offered
ALPN/HTTP2 h2, http/1.1 (offered)
Done 2021-09-17 09:37:04 [ 94s] -->> 20.43.111.112:443 (duckduckgo.com) <<--
real 1m34.440s
user 0m5.460s
sys 0m2.712s
And this was with NXCONNECT:
$ time NXCONNECT=localhost:0 ./testssl.sh -p duckduckgo.com
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Start 2021-09-17 10:24:18 -->> 20.43.111.112:443 (duckduckgo.com) <<--
rDNS (20.43.111.112): --
It's been running for 5 hours now, with no sign of halting!
I just tried the NXCONNECT version again, and it was quicker this time:
$ time NXCONNECT=localhost:0 ./testssl.sh -p duckduckgo.com
###########################################################
testssl.sh 3.1dev from https://testssl.sh/dev/
(fe6c22f 2021-09-15 09:53:20 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.1.1f 31 Mar 2020" [~98 ciphers]
on CCM310:/usr/bin/openssl
(built: "Aug 23 17:02:39 2021", platform: "debian-amd64")
Start 2021-09-17 15:29:16 -->> 20.43.111.112:443 (duckduckgo.com) <<--
rDNS (20.43.111.112): --
Service detected: HTTP
Testing protocols via sockets except NPN+ALPN
SSLv2 not offered (OK)
SSLv3 not offered (OK)
TLS 1 not offered
TLS 1.1 not offered
TLS 1.2 offered (OK)
TLS 1.3 offered (OK): final
NPN/SPDY not offered
ALPN/HTTP2 h2, http/1.1 (offered)
Done 2021-09-17 15:29:23 [ 17s] -->> 20.43.111.112:443 (duckduckgo.com) <<--
real 0m18.875s
user 0m5.781s
sys 0m4.850s
It looks like it's quicker with NXCONNECT, except for one time when it hung for 5 hours. Perhaps duckduckgo dropped the connection, and testssl.sh didn't notice?
The occasion where it hung is not something we can use for production unless it was a one time thing.
If you like you can exchange localhost by 127.0.0.1 and later the port by something else and try but I am afraid in general we need good data as a basis.