Error while upgrading cnx to http

[verbal666]

Describe the goal

ws server is the same on both env. Connect wstunnel thru a proxy chains

• in a TEST environment, it works [wstunnel client]-> local http proxy -> remote http proxy -> remote socks5 proxy -> remote http proxy -> wstunnel server
• in a PROD environment, quite same chaining, got an error 🤔🤔🤔

Testing with a simple curl -kL https://url -x socks5h://wstunnel_ip:socks5_ip ,

• in TEST works perfect
• in PROD i got errors Error while upgrading cnx to http 🥲

Describe what does not work

Client side i have no errors, apparently, wstunnel starts and starts listening at its socks5. Server side i got a strange error,

ERROR cnx{peer="1.2.3.4:31734"}: wstunnel::tunnel::server::server: Error while upgrading cnx to http: hyper::Error(Io, Custom { kind: UnexpectedEof, error: "connection closed before reading preface" })

Maybe production http proxy filter and alter https/ws packets? 🥲

Describe your wstunnel setup

server wss://local_ip:443 --tls-certificate \path_to\fullchain.pem --tls-private-key \path_to\privkey.pem --restrict-http-upgrade-path-prefix my_private_prefix --remote-to-local-server-idle-timeout 30m --websocket-ping-frequency-sec 600 --log-lvl DEBUG

 INFO cnx{peer="1.2.3.4:31734"}: wstunnel::tunnel::server::server: Accepting connection
 INFO cnx{peer="1.2.3.4:31734"}: wstunnel::tunnel::server::server: Doing TLS handshake
DEBUG cnx{peer="1.2.3.4:31734"}: rustls::server::hs: decided upon suite TLS13_AES_256_GCM_SHA384
DEBUG cnx{peer="1.2.3.4:31734"}: rustls::server::hs: Chosen ALPN protocol [104, 50]
ERROR cnx{peer="1.2.3.4:31734"}: wstunnel::tunnel::server::server: Error while upgrading cnx to http: hyper::Error(Io, Custom { kind: UnexpectedEof, error: "connection closed before reading preface" })

Desktop (please complete the following information):

Chaining many OS ==> Linux (Ubuntu / Debian / Alma) and also Windows

[erebe]

Are you starting your wstunnel client to use http2 ?

Because the server see an incoming TLS connection with Chosen ALPN protocol [104, 50] which means h2 (http2). While for websocket the ALPN should be http/1.1

You can use TRACE log level on the server to get more info.

[verbal666]

I'll try tracing 👍

But i think it's the production proxy (it has also a firewall onboard!) which filters or alters the TLS communication with the ws server, and so the connection is for some networking reason not going well.

Considering the test proxy has no filters/firewall and it's full open, and ws client connects well 🤷‍♀️

PS. how can i force not using HTTP2? Can't see parameters client side 🤔🤔🤔

Attached a clean session with a single curl get. (hoping have removed all sensible data 🙏🙏🙏)

[cut]

[erebe]

Ha it seems you start your client with http2 as transport protocol

wstunnel client -L socks5://127.0.0.1:1080 --connection-min-idle 0 --tls-sni-override my.domain.com --http-upgrade-path-prefix my_path https://my.domain.com:443 --tls-certificate /etc/certs/fullchain.pem --tls-private-key /etc/certs/privkey.pem --tls-verify-certificate -p proxy:port --log-lvl TRACE

 https://my.domain.com:443

to use websocket as transport, you need to change it to

 wss://my.domain.com:443

It should work better after that

[verbal666]

Gosh! You're perfect right. Didn't noticed i started remote to https 🤦‍♂️force of habit 😉 I'll try again soon with remote wss:// . But i really think the proxy/firewall has something which changes headers/packets 🥲 Thanks 👍👍👍

[verbal666]

Surely production proxy/fw is filtering 👎👎👎

TEST PROXY == OK == tunnel & next curl went ok PROD PROXY == KO == alpn: None / DEBUG cnx{peer="my_remote_ip:32094"}: rustls::common_state: Sending warning alert CloseNotify

[cut]

I'll find another way to connect the tunnel 🤷‍♀️

[erebe]

Indeed that's strange, it seems you never receive data server side. To investigate more, I would need a network capture.

export SSLKEYLOGFILE=/tmp/tls_keys.txt
wstunnel server ... wss://local_ip:443

# in another terminal 
sudo tcpdump -i any port 443 -w capture.pcap -s 65535

and you can send me the /tmp/tls_keys.txt and capture.pcap files for a ko request

[verbal666]

Not so strange. Prod Proxy (HTTPS) and its firewall has many professional filters and tls inspectors. Surely it changes the packets flow so ws can't understand what's going on 🤷‍♀️🤷‍♀️🤷‍♀️

wstunnel works perfect, it's proxy who alters its flow 🥲

Don't mind about it. Eventually i'll send the tcpdump, in other case ok so 🤷‍♀️

[bytejedi]

@verbal666 @erebe You can try use a public CA like letsencrypt. Some firewall block selfsigned cert or some specific SNI. If public CA is not work, you can try selfsigned a cert that SNI is localhost. Good luck

[verbal666]

Hi. Still have my certs signed with Let'sEncrypt 👍 I manage 3 domains, all with L/E signed certs. My HTTPS proxy blocks all not signed certs by years. I can with no problems navigate all my webservers thru HTTPS proxy, so it does not blocks HTTPS/TLS negotiate.

HTTPS proxy has many many professional filters and also provides its certs for authentication, so it changes original certs with its own... so, something in the middle [proxy] (also seeing what i'm doing 😤) broke the connection.

Thanks.

[verbal666]

I also tried a session with stunnel, which works more less like yours, perfect on any of my clients, both Linux and Windows, and thru that proxy it does not work. The proxy/firewall filters all data and breaks any not HTTP traffic. Probably a very deep DPI interrupts all traffic! 🤷‍♀️Le roi est mort, vive le roi!!! 🤷‍♀️🤷‍♀️ PS. can't use redsocks or shadowsocks or similar, since connection to public ip:port is denied 👎

[erebe]

That's sad, maybe your proxy forbid completely websocket. You can give a try by visiting this website https://websocketstest.com/

Out of curiosity, in which kind of environment are you working in to have such strong firewall ?

[verbal666]

Yes, not properly sad 😉but frustrating. Can't talk much 🤭but it's a very strong fw/av, our Company migrated the previous proxy to an external service provider, and they invest much on security (correctly!!! 👍), but censorship policy is very very frustrating!!! We -developers- can't neither download apache/tomcat/tools or others bins/sources from the inet, since 1) or target is denied (access is blocked) 2) or target starts downloading and then proxy cuts and truncate the final download file to 10%, responding 200, giving a corrupted file 🥲🥲🥲👎 3) any "strange" CONNECT is monitored and a notification runs on every event of such type 🤦‍♂️🤦‍♂️🤦‍♂️

Just to understand,

On the TCP Listener (Port 443), a Client (IP address 1.2.3.4, Host name "1.2.3.4", Port number 47078) has connected.
For the client (IP address: 1.2.3.4, host name: "1.2.3.4", port number: 47078), connection "CID-1-9EDE0C5552" has been created.
SSL communication for connection "CID-1-9EDE0C5552" has been started. The encryption algorithm name is "TLS_AES_256_GCM_SHA384".
Connection "CID-1-9EDE0C5552" terminated by the cause "A client which is non-SoftEther VPN software has connected to the port." (code 5).
Connection "CID-1-9EDE0C5552" has been terminated.
The connection with the client (IP address 1.2.3.4, Port number 47078) has been disconnected.

With same identical config i can connect both to wstunnel and to my vpn outside that proxy/fw/av 🤷‍♀️

So, HAIL TO THE KING 🤷‍♀️🤷‍♀️🤷‍♀️😤

🤷‍♀️

[verbal666]

Anyway, just to let you know 🥲🥲🥲

WebSockets seem to
Not Work for you :(

Environment
WebSockets supported YES
HTTP Proxy NO

WebSockets (Port 443, SSL)
Connected No ❌

🤷‍♀️🤷‍♀️🤷‍♀️