Ratchet
Ratchet copied to clipboard
ratchetphp
Javascript send() event firing onclose() event (browser) after 12 hours (or so) of connection with the websocket (but the server is still up/running)
Well, folks, this is what I've experienced for practically one month or so: I have a websocket service running via supervisor on port 8080, and it works perfectly with my code implementation (PHP/Javascript/Codeigniter)... Yes, it works well, but just until PAST 10 OR 12 HOURS (GENERALLY THE FOLLOWING DAY). When I'm gonna check the websocket service itself, okay, it's there... Running on port 8080, okay... The issue seems to be on client side (at least apparently, because I also suspect that there may be some hinder on server side). Checking my console (Javascript logs), and I see that:
- It really connects (onopen() is fired normally)
- onmessage() works
- But... when the algorhythm tries to do anything with websocket.send(), the connection is closed immediately (note: onerror() is not fired, but onclose() is and weirdly no error). This happens on client side (browser)
- I have ev (installed via pecl) installed and running there, I did it yesterday, but the issue seems to persist.
- I'm using OriginCheck, but I believe it has no relation at all with the issue
- In Ningx, I'm using a server block similar to this:
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream ws-backend {
# server websocket.your-site.com:8080;
# server localhost:8080;
# server 135.659.25.124:8080;
server 127.0.0.1:8080;
}
#If you want to force HTTPS, use the server block below:
server {
listen 80;
listen [::]:80;
server_name websocket.your-site.com
return 301 https://$host$request_uri;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name websocket.your-site.com;
#Start the SSL configurations
#ssl on;
ssl_protocols TLSv1.2;
# Fix The Logjam Attack.
ssl_ciphers EECDH+AESGCM:EDH+AE...; #only for example purposes
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/ssl/dh2048_param.pem;
ssl_certificate /etc/pathto/websocket.your-site.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/pathto/websocket.your-site.com/privkey.pem; # managed by Certbot
location / {
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $http_host;
proxy_pass http://ws-backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
}
- As you can see above, I created a subdomain just for the websocket thing. and I'm using
Connection $connection_upgradealways. If I useConnection "", I simply will not be able to connect to the websocket any longer, it outputs:WebSocket connection to 'wss://websocket.your-site.com/' failed:. But... Of course, this doesn't have anything to do with the issue either - In nginx.conf, I set
worker_processes auto;,worker_rlimit_nofile 500000;,worker_connections 1024;... At least so far... But I think those numbers are more than enough - I checked the file descriptors limit and I solved to increase (absurdly) some values:
fs.file-max = 2097152there in /etc/sysctl.conf,root hard nofile 500000,oot soft nofile 500000, there in /etc/security/limits.conf,session required pam_limits.sothere in /etc/pam.d/common-session - I'm using PDO and the database connection settings have been placed inside the construct method of the websocket app main class
- The port 8080 (which I use to connect the websocket) is okay and enabled by
sudo ufw allow 8080/tcp - These are my server specs: 16GB RAM, 400GB SSD, 6vCPU cores, 32TB bandwidth (unlimited incoming traffic), PHP 8.0.14, Ubuntu 20.04 LTS, Nginx 1.18.0
- In sum, I just dunno what on Earth is causing this. In truth, I can't take it any longer, lol.
Originally posted by @roguitar88 in https://github.com/ratchetphp/Ratchet/issues/328#issuecomment-1010562500
Start by replacing PDO with https://github.com/friends-of-reactphp/mysql
Well, in addition to that, I've figured out a few things that may help us. Supervisor has shown me this:
-
PHP Warning: Attempt to read property "type" on null in server.php on line 68(where server.php has that application class) -
PHP Warning: Packets out of order. Expected 1 received 0. Packet size=145 in server.php on line 277
On line 68, we have:
....
public function onMessage(ConnectionInterface $from, $data) {
$data = json_decode($data);
$type = $data->type; // This is the line 68
....
}
On line 277, we have:
....
$sql = $this->db->prepare('SELECT * FROM user WHERE cpf = :cpf'); // Using PDO as I said
$sql->bindParam(':cpf', $cpf, PDO::PARAM_STR);
$sql->execute(); // This is the line 277
$count = $sql->rowCount();
....
And the connection to the database is being made this way (there in server.php):
....
protected $clients;
protected $db;
public function __construct() {
$datenbank = true;
if ($datenbank) {
$c_host = "localhost";
$c_user = "hotpotato";
$c_pass = "kissmycucumber";
$c_datenbank = "my_database";
} else {
$c_host = "localhost";
$c_user = "root";
$c_pass = "";
$c_datenbank = "my_database";
}
$this->db = new PDO("mysql:host=".$c_host.";dbname=".$c_datenbank."", $c_user, $c_pass, array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET CHARACTER SET utf8, NAMES utf8"));
$this->db->setAttribute(PDO::ATTR_ERRMODE,PDO::ERRMODE_EXCEPTION);
$this->clients = new \SplObjectStorage;
}
....
Start by replacing PDO with https://github.com/friends-of-reactphp/mysql
From that last warning I've just pointed out, I strongly suspect that PDO may be one of the causes of it (not to say the major cause). Speaking of which, I've noticed this:
- The connection (browser-side) closes only when send() events are fired towards requests to the database (via PDO). But strangely or not, if that is really the case, why in the first 10 or 12 hours, those kinds of requests work normally and then not? (Check again the second warning, which, by the way, I haven't understood quite well).
As for the first warning, I really don't understand why $data->type is coming null, since all my send() event codes really have 'type' as element of the array in JSON.stringify, like so, for example (in Javascript):
....
socket.send(
JSON.stringify({
'type': 'verify_cpf', // It's defined
// 'user_from': this.userFrom,
'cpf': value
})
)
....
Anyways, In order to avoid that warning, I decided to do this:
....
public function onMessage(ConnectionInterface $from, $data) {
$data = json_decode($data);
if ($data->type != NULL) { // I wrapped the whole logic in this condition
$type = $data->type;
....
Besides all that, I'd like to mention something that may not have any relation with the problem, but that I guess it may be important to let you know here, too: I had to alter /vendor/cboden/ratchet/src/Ratchet/Http/OriginCheck.php, because I was having this issue here:
PHP Warning: Undefined array key 0 in /vendor/cboden/ratchet/src/Ratchet/Http/OriginCheck.php on line 38
I had to alter it from this:
....
public function onOpen(ConnectionInterface $conn, RequestInterface $request = null) {
$header = (string)$request->getHeader('Origin')[0]; // Line 38
$origin = parse_url($header, PHP_URL_HOST) ?: $header;
if (!in_array($origin, $this->allowedOrigins)) {
return $this->close($conn, 403);
}
return $this->_component->onOpen($conn, $request);
}
....
to this:
....
public function onOpen(ConnectionInterface $conn, RequestInterface $request = null) {
if (!empty($request->getHeader('Origin'))) { // I added this condition
$header = (string)$request->getHeader('Origin')[0];
$origin = parse_url($header, PHP_URL_HOST) ?: $header;
if (!in_array($origin, $this->allowedOrigins)) {
return $this->close($conn, 403);
}
}
return $this->_component->onOpen($conn, $request);
}
....
I'd just like to emphasize that the websocket connection issue was ocurring even before this alteration. That's why I believe this doesn't have any relation with problem. Anyways, it doesn't hurt to report that. Maybe I'm mistaken. I know you're gonna say it's counterproductive, because if I run something like composer update, those alterations would go down the drain.
But forgetting that and coming back to the original problem, what do you believe is really causing this? Note: those are just warnings. And overnight, I found only one ocurrence of each reading throughout the logs. So, what I mean is that those warnings are not being fired on account of the send() events, but sporadically and rarely.
Well, in addition to that, I've figured out a few things that may help us. Supervisor has shown me this:
PHP Warning: Attempt to read property "type" on null in server.php on line 68(where server.php has that application class)PHP Warning: Packets out of order. Expected 1 received 0. Packet size=145 in server.php on line 277On line 68, we have:
.... public function onMessage(ConnectionInterface $from, $data) { $data = json_decode($data); $type = $data->type; // This is the line 68 .... }On line 277, we have:
.... $sql = $this->db->prepare('SELECT * FROM user WHERE cpf = :cpf'); // Using PDO as I said $sql->bindParam(':cpf', $cpf, PDO::PARAM_STR); $sql->execute(); // This is the line 277 $count = $sql->rowCount(); ....And the connection to the database is being made this way (there in server.php):
.... protected $clients; protected $db; public function __construct() { $datenbank = true; if ($datenbank) { $c_host = "localhost"; $c_user = "hotpotato"; $c_pass = "kissmycucumber"; $c_datenbank = "my_database"; } else { $c_host = "localhost"; $c_user = "root"; $c_pass = ""; $c_datenbank = "my_database"; } $this->db = new PDO("mysql:host=".$c_host.";dbname=".$c_datenbank."", $c_user, $c_pass, array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET CHARACTER SET utf8, NAMES utf8")); $this->db->setAttribute(PDO::ATTR_ERRMODE,PDO::ERRMODE_EXCEPTION); $this->clients = new \SplObjectStorage; } ....Start by replacing PDO with https://github.com/friends-of-reactphp/mysql
From that last warning I've just pointed out, I strongly suspect that PDO may be one of the causes of it (not to say the major cause). Speaking of which, I've noticed this:
- The connection (browser-side) closes only when send() events are fired towards requests to the database (via PDO). But strangely or not, if that is really the case, why in the first 10 or 12 hours, those kinds of requests work normally and then not? (Check again the second warning, which, by the way, I haven't understood quite well).
As for the first warning, I really don't understand why $data->type is coming null, since all my send() event codes really have 'type' as element of the array in JSON.stringify, like so, for example (in Javascript):
.... socket.send( JSON.stringify({ 'type': 'verify_cpf', // It's defined // 'user_from': this.userFrom, 'cpf': value }) ) ....Anyways, In order to avoid that warning, I decided to do this:
.... public function onMessage(ConnectionInterface $from, $data) { $data = json_decode($data); if ($data->type != NULL) { // I wrapped the whole logic in this condition $type = $data->type; ....Besides all that, I'd like to mention something that may not have any relation with the problem, but that I guess it may be important to let you know here, too: I had to alter /vendor/cboden/ratchet/src/Ratchet/Http/OriginCheck.php, because I was having this issue here:
PHP Warning: Undefined array key 0 in /vendor/cboden/ratchet/src/Ratchet/Http/OriginCheck.php on line 38I had to alter it from this:
.... public function onOpen(ConnectionInterface $conn, RequestInterface $request = null) { $header = (string)$request->getHeader('Origin')[0]; // Line 38 $origin = parse_url($header, PHP_URL_HOST) ?: $header; if (!in_array($origin, $this->allowedOrigins)) { return $this->close($conn, 403); } return $this->_component->onOpen($conn, $request); } ....to this:
.... public function onOpen(ConnectionInterface $conn, RequestInterface $request = null) { if (!empty($request->getHeader('Origin'))) { // I added this condition $header = (string)$request->getHeader('Origin')[0]; $origin = parse_url($header, PHP_URL_HOST) ?: $header; if (!in_array($origin, $this->allowedOrigins)) { return $this->close($conn, 403); } } return $this->_component->onOpen($conn, $request); } ....I'd just like to emphasize that the websocket connection issue was ocurring even before this alteration. That's why I believe this doesn't have any relation with problem. Anyways, it doesn't hurt to report that. Maybe I'm mistaken. I know you're gonna say it's counterproductive, because if I run something like
composer update, those alterations would go down the drain.But forgetting that and coming back to the original problem, what do you believe is really causing this? Note: those are just warnings. And overnight, I found only one ocurrence of each reading throughout the logs. So, what I mean is that those warnings are not being fired on account of the send() events, but sporadically and rarely.
I have the exact same problem. First onOpen message always works, I have a ping socket command which also calls the socket once and response is always fine. Now if I run this socket for a few days, some of the system users starts to get this closed socket problem. Problem is that this socket closes right after the first greeting onOpen message. And only restart helps.
Have you found any solution?
