websocketpp
websocketpp copied to clipboard
zaphoyd
Performance issue with WSS connection
Hi,
I noticed a strange performance issue with Websockets over TLS. I'm using Websocket++ 0.5.1 on Windows, but I didn't see anything in the changelog indicating that the problem might have been addressed in the meantime.
My application uses Websocket++ on the client side, the server is IIS 8.5. When I talk to the server over an unencrypted connection, there is no problem. Over an TLS channel, however, sending a message from the client to the server takes about 50ms longer than expected.
With the help of Wireshark and some precarious server settings, I've been able to decrypt the communication. The client wants to send a Text payload of 91 bytes to the server. Instead of sending it in a single ethernet frame, the communication is as follows:
-
The client sends a frame to the server which contains only 6 bytes, namely the Websocket Header. As an example, the Websocket header looks like 0x81 DB 45 E0 62 85. In other words, at this moment, it is already known that the payload is 91 bytes; the FIN bit is set to 1.
-
The server acknowledges frame 1. It takes the server (about) 50ms to acknowledge Frame 1, which is where the performance penalty comes from.
-
After receiving the ACK, the client immediately sends the actual payload.
In the unencrypted case, the client sends the whole payload at once.
Is this a known issue? Is there a workaround? I should add that I use Websocket++ only indirectly (I use SignalR, which in turn uses the C++ REST SDK, but since the fragmentation is always made directly after the Websocket-Header, my first suspicion would be the Websocket-Layer...)
WebSocket++ issues the header and payload writes separately because the WebSocket frame header is variable length and putting them in one buffer would require an extra copy of the payload on all messages. When using a supported transport (Asio is one of them) these writes are supposed to be coalesced by that underlying layer (likely Asio in your this case).
It sounds like this coalescing is happening for unencrypted connections but not encrypted ones. I'm pretty sure my testing from that period of development indicated that Asio coalesced the header and payload into a single TLS segment on my testing machines (which are largely Linux and BSD based).
It sounds like some more testing needs to be done with Asio's scatter-gather I/O on Windows when using a TLS wrapper.
I can confirm that the underlying layer is indeed Asio (Boost 1.58)
I also see issues with WebSocket SSL under Windows environment. Here comes my test results in comparison Unsecured WebSockets vs WebSocket SSL:
WebSockets Server address: 127.0.0.1 Server port: 4444 Server uri: ws://127.0.0.1:4444 Working threads: 1 Working clients: 1 Messages to send: 10000 Message size: 32
Send time: 8.071 ms Send bytes: 320000 Send messages: 10000 Send bytes throughput: 39647332 bytes per second Send messages throughput: 1238979 messages per second Receive time: 235.850 ms Receive bytes: 320000 Receive messages: 10000 Receive bytes throughput: 1356789 bytes per second Receive messages throughput: 42399 messages per second Errors: 0
WebSocket SSL Server address: 127.0.0.1 Server port: 5555 Server uri: wss://127.0.0.1:5555 Working threads: 1 Working clients: 1 Messages to send: 10000 Message size: 32
Send time: 9.180 ms Send bytes: 320000 Send messages: 10000 Send bytes throughput: 34857476 bytes per second Send messages throughput: 1089296 messages per second Receive time: 2.402 s Receive bytes: 320000 Receive messages: 10000 Receive bytes throughput: 133187 bytes per second Receive messages throughput: 4162 messages per second Errors: 0
It is a known asio ssl issue. See here: https://github.com/boostorg/asio/issues/100
If you are interested in the asio code see here: https://github.com/darwin/boost/blob/master/boost/asio/ssl/detail/openssl_stream_service.hpp#L378-L380
hi, I face the same issue, use websocketpp 0.8.2 as websocket client , transport layer is asio (boost 1.73.0), in none tls mode send message to server and receive response in 0.3 ms, but need 40+ms in tls mode, it seems asio won't fix this issue since two years passed, is there any workaround?
Blah. It does seem that this is not getting fixed on the Asio (or possibly even openssl end). Seems like the recommend solution is in fact for end users to just not use efficient scatter/gather IO. That is unfortunate, but it is something that can be implemented at the WebSocket++ level.
hi, zaphoyd, thank you for the reply, I'm not familliar with Asio, to avoid this issue I choose copy header and payload into one buffer, it's works but inefficient, and I don't know weather it will bring in any other problems, so it would be nice if you can do some update on this issue. anyway, thank you for sharing such great project.
To fix this we might introduce a buffer type into the config class so the user can provide its own flat buffer. like it is done on the mentioned work around above.
boostorg/asio#100 has been marked as closed. Is there anyone can confirm this issue also been solved?
Hi,
I noticed a strange performance issue with Websockets over TLS. I'm using Websocket++ 0.5.1 on Windows, but I didn't see anything in the changelog indicating that the problem might have been addressed in the meantime.
My application uses Websocket++ on the client side, the server is IIS 8.5. When I talk to the server over an unencrypted connection, there is no problem. Over an TLS channel, however, sending a message from the client to the server takes about 50ms longer than expected.
With the help of Wireshark and some precarious server settings, I've been able to decrypt the communication. The client wants to send a Text payload of 91 bytes to the server. Instead of sending it in a single ethernet frame, the communication is as follows:
- The client sends a frame to the server which contains only 6 bytes, namely the Websocket Header. As an example, the Websocket header looks like 0x81 DB 45 E0 62 85. In other words, at this moment, it is already known that the payload is 91 bytes; the FIN bit is set to 1.
- The server acknowledges frame 1. It takes the server (about) 50ms to acknowledge Frame 1, which is where the performance penalty comes from.
- After receiving the ACK, the client immediately sends the actual payload.
In the unencrypted case, the client sends the whole payload at once.
Is this a known issue? Is there a workaround? I should add that I use Websocket++ only indirectly (I use SignalR, which in turn uses the C++ REST SDK, but since the fragmentation is always made directly after the Websocket-Header, my first suspicion would be the Websocket-Layer...)
Why the client had to delay sending the 2nd frame until receiving the ACK of the 1st frame from the server?
