OpenVPN
openvpn tcp protocol over frp forwarding issues
My problem is that openvpn log shows frequently "connection reseting", "client restarting"
as follows:
Jun 11 12:29:42 k8s-pre-node01 openvpn: TCP connection established with [AF_INET]172.31.1.46:50990
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50990 Connection reset, restarting [0]
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50990 SIGUSR1[soft,connection-reset] received, client-instance restarting
Jun 11 12:29:42 k8s-pre-node01 openvpn: TCP connection established with [AF_INET]172.31.1.46:50992
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50992 Connection reset, restarting [0]
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50992 SIGUSR1[soft,connection-reset] received, client-instance restarting
Jun 11 12:29:42 k8s-pre-node01 openvpn: TCP connection established with [AF_INET]172.31.1.46:50994
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50994 Connection reset, restarting [0]
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50994 SIGUSR1[soft,connection-reset] received, client-instance restarting
Jun 11 12:29:42 k8s-pre-node01 openvpn: TCP connection established with [AF_INET]172.31.1.46:50996
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50996 Connection reset, restarting [0]
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50996 SIGUSR1[soft,connection-reset] received, client-instance restarting
Jun 11 12:29:42 k8s-pre-node01 openvpn: TCP connection established with [AF_INET]172.31.1.46:50998
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50998 Connection reset, restarting [0]
Jun 11 12:29:42 k8s-pre-node01 openvpn: 172.31.1.46:50998 SIGUSR1[soft,connection-reset] received, client-instance restarting
here is my openvpn configuration:
local 192.168.7.4
port 1941
proto tcp
dev tun
ca /etc/openvpn/3.0.7/pki/ca.crt
cert /etc/openvpn/3.0.7/pki/issued/server.crt
key /etc/openvpn/3.0.7/pki/private/server.key
dh /etc/openvpn/3.0.7/pki/dh.pem
server 192.168.41.0 255.255.255.0
ifconfig-pool-persist ipp.txt
push "route 172.30.1.0 255.255.255.0"
keepalive 10 120
tls-auth /etc/openvpn/3.0.7/pki/ta.key 0
cipher AES-256-CBC
comp-lzo
persist-key
persist-tun
status openvpn-status.log
verb 3
reneg-sec 0
here is my frp configuration:
serverAddr = "123.8.141.2"
serverPort = 25056
auth.method = "token"
auth.token = "xxxxxxxxxxxxxxxxxxxxxxxxxx"
transport.tls.enable = true
transport.tls.certFile = "/usr/local/tls-frp/client.crt"
transport.tls.keyFile = "/usr/local/tls-frp/client.key"
transport.tls.trustedCaFile = "/usr/local/tls-frp/ca.crt"
[[proxies]]
name = "tfdx"
type = "tcp"
localIP = "192.168.7.4"
localPort = 1941
remotePort = 25058
Actually, when i change openvpn tcp to udp, this problem disappears. I don't know why openvpn tcp over frp does not work.
This sounds more like an frp problem, doesn't it? I don't know frp but maybe that is some kind of health check?
This sounds more like an frp problem, doesn't it? I don't know frp but maybe that is some kind of health check?
frp official project: https://github.com/fatedier/frp
I guess we encontered a classical problem named "tcp over tcp"，which can magnify tcp retransmission。If you doubt it's an frp bug, I would like to believe neither openvpn nor frp has any bug. Tcp meltdown is a very old myth. I am not sophisticated at this area.
And, i have made a lot of test, frp forwarding sshd works fine, frp forwarding mysql works fine, which all belong to "tcp over tcp".
refer: https://news.ycombinator.com/item?id=25080693
Tcp meltdown is a very old myth.
I don't think it is a myth.
I guess we encontered a classical problem named "tcp over tcp"，which can magnify tcp retransmission。
I am not sure how you arrived at the conclusion that you are using tcp over tcp. I though frp was just a proxy.
And, i have made a lot of test, frp forwarding sshd works fine, frp forwarding mysql works fine, which all belong to "tcp over tcp".
This is not TCP over TCP.
Tcp meltdown is a very old myth.
I don't think it is a myth.
I guess we encontered a classical problem named "tcp over tcp"，which can magnify tcp retransmission。
I am not sure how you arrived at the conclusion that you are using tcp over tcp. I though frp was just a proxy.
And, i have made a lot of test, frp forwarding sshd works fine, frp forwarding mysql works fine, which all belong to "tcp over tcp".
This is not TCP over TCP.
OK，thank you anyway. I still get nothing about this problem.
OK，thank you anyway. I still get nothing about this problem.
Because where the problem sits is far from clear. Arne was hoping to get some detail about what frp is doing, but so far we know nothing about it.
What the OpenVPN log says simply means "the TCP connection died".
One next step might be to sniff some traffic (tcpdump/wireshark) between OpenVPN and the proxy and see who is resetting and possibly try to infer why. Maybe frp has also some logs which with the right verbosity may tell us something?
Note: none of the above is yet about OpenVPN per se, but we just need some information to at least speculate about the problem.
If OpenVPN logs "connection reset", something in the network is breaking the TCP connection - the log shows that it was possible to reach the other endpoint, and then "something" broke it. From within OpenVPN, this is all the information we have. So, as @ordex said, you need to figure out what FRP is doing - ideally by running tcpdump on both ends of that FRP thing, to see what TCP packets are going in and out, and who is breaking the session.
Since FRP is a proxy, it might be a problem between the other side and the OpenVPN peer you try to talk to - proxies can not know in advance if the connection on the other side will succeed. So you connect to the proxy, the other end tries to connect to the server, that fails (for whatever remote reason), and then the proxy needs to reset the client session - nothing else it can do.
So verify that the FRP setup can actually reach the intended OpenVPN peer on the configured TCP port, and that there is nothing on the OpenVPN side that refuses the connection.
This sounds more like an frp problem, doesn't it? I don't know frp but maybe that is some kind of health check?
I am sorry, you are totally right, this is health check on cloud ELB which i've never noticed.
@schwabe @ordex @cron2 thanks for your guidence, i am a little student, i still have a long journay to study.