sonertari
Question: Divert socket seems to not be forwarding traffic back to original client
I'm working on a modification to sslproxy to use addresses provided using the haproxy PROXY v1 protocol from the initial client TCP connection to allow easier redirection and load distribution without the need for complicated iptables which isn't available in all environments.
My test environment is the following
┌───────────┐
│sslproxy │
└─────▲─────┘
│
│
│
┌─────▼─────┐
┌────────┐ │Python │ ┌───────┐
│ Client ◄───────►Redirector ◄───────►Server │
└────────┘ └───────────┘ └───────┘
All connections are made successfully; sslproxy reports TLS negotiation to client and server; the divert socket in the python redirector shows all data, unencrypted, and has no issues passing messages. tcpdump on the lo interface shows diverted responses coming back from sslproxy are received and sent back to sslproxy as well. However, sslproxy never passes the encrypted data back to the client socket.
I'm really struggling to debug this and would appreciate some thoughts.
The main code change is to prototcp_fd_readcb. I've added a handler that will peek the receive buffer, parse the PROXY header if present, and override srcaddr/dstaddr if the header is parsed successfully. In pxy_conn_init I have to bypass dstaddr handling as well.
The only odd thing I see in the packet capture is that there is 2 TCP connections sent from sslproxy to the divert port. The first remains unused/idle while the second receives the SSLproxy header and is where the decrypted traffic is sent back to.
Found the problem. I haven't traced the reason, however, BEV_EVENT_CONNECTED fires twice for srvdst when autossl is used. This causes prototcp_bev_eventcb_connected_srvdst to be called twice and subsequently prototcp_setup_dst to be called twice. The first call initiates a divert connection which is apparently maintained as the socket for dst->src. The second call leaks the original dst bev and initiates a second connection which is used for src->dst. Avoiding the second setup appears to resolve the issue, but I can't say all the ramifications.
You sound like you know what you're doing, so I assume you already know that SSLproxy is designed to divert the decrypted packets to a listening program, and to expect the possibly modified packets back from the listening program, and then to reencrypt the packets and send them to the Server.
Given that, your diagram looks strange to me, as I don't know your changes to the SSLproxy source code, because your diagram shows your Python Redirector and sslproxy in switched locations, compared to my Mode of Operation diagram in README.
If your diagram is correct, your Python Redirector accepts connections from Client, sends them to SSLproxy? But the logs you have provided do not seem like it, because sslproxy is able to determine the original src and dst addresses, which wouldn't be possible if it is your Python Redirector who accepts connections from Client, because sslproxy finds those addresses from the NAT engine in the system.
As is obvious, I cannot understand your setup and what you are trying to achieve. Can you explain what you expect sslproxy to do exactly here? Perhaps all I need to know is what sslproxy is supposed to do in your setup.
Btw, it's great that you were able to figure out how to use verbose logging in sslproxy.
My primary goal is to create a version of sslproxy that can process connections forwarded via a load balancer.
The general idea would be to use haproxy in transparent intercept mode and round robin the connections to multiple sslproxy instances. The catch is that I want to maintain knowledge of the true source and destination for logging and I don't want a ton of complicated NAT infrastructure.
The python redirector is just a test harness I've built to facilitate code modifications without the need for a fully deployed infrastructure.
The general code changes I'm working through:
- Add support for sslproxy to receive the haproxy PROXYv1 header
- Add support for sslproxy to send the haproxy PROXYv1 header if the header was received from the client side and a target IP is configured
From the sslproxy perspective: haproxy H connects to sslproxy IP S and sends client source C and destination D in PROXY header sslproxy establishes connection to upstream proxy U and sends client source C and destination D in PROXY header sslproxy establishes divert connection to inspector and sends client source C and destination D in SSLProxy header Upstream proxy U connects to server D using forged source C
I've been working through the sslproxy state machine quite a bit and it looks like there may be some unintended duplicate calls to prototcp_setup_dst when autoproxy is used and detects an SSL connection. Both seem to come through protoautossl_bev_eventcb --> prototcp_bev_eventcb_srvdst. It seems to me there is a second BEV_EVENT_CONNECTED event when SSL finishes. I haven't found a safe way to avoid the event, the best I can do is avoid the leak of ctx->srvdst.