node-mariasql
node-mariasql copied to clipboard
mscdex
Assertion `status == 0' failed in cb_poll()
Sometimes one node-worker exits with the following error:
node: ../src/binding.cc:581: static void Client::cb_poll(uv_poll_t*, int, int):
Assertion `status == 0' failed.
Possibly this happens after MySQL-Server closes the connection after idle timeout (60 seconds in our case).
This was the state when the assertion failed:
- status = -1
- uv_last_error(uv_default_loop()).code = EBADF
- obj->state = STATE_QUERYING (for STATE_CONNECTING this case is handled some lines above)
The error is often (but not always) reproducable, sometimes a normal error-event is thrown with message "Disconnected from the server".
Maybe this relates to Issue #55 (Uncatchable Error "Not ready to query").
Ah sorry, no, on Linux (Debian 7), Node 0.10.29 with the latest stable (0.1.21) mariasql version.
I reproduce this currently with a chance of ~ 30-50% (otherwise a "normal" error is thrown, which is ok):
- start the node-service
- do one or two HTTP-requests triggering a database query
- wait until MySQL-Server closes connection
- the error is shown in console and the worker dies
Just wanted to bump this to say that I've also seen the exact same problem recently.
Have been running my application fine for a long time. Then we moved MySQL from localhost onto another machine over LAN, and have started noticing this issue. The timeouts on the MySQL server are set quite generously, though I wouldn't rule out that being a possible cause.
Running Node 0.10.35 with 0.1.21 mariasql on Ubuntu/Debian.
Thanks for the module by the way!
When you say "idle timeout" are you referring to the wait_timeout value on the server-side? I am working on a rewrite and its keepalive functionality currently and would like to see if I can reproduce this.
Just had the same issue today, have been using the mariasql for a long time in production on bare metal at a private datacenter and never had the issue before. But recently (a month) moved to Amazon's RDS, and it just happened for the first time. Gives me an impression my situation is similar to @zingaburga
The actual error is probably something low-level (obviously since it's at the uv level). So Mariasql, may not be guilty at all. But just for reference I'm posting here :)
Somewhat hoping that upgrading to Node 0.12 will solve the issue...
Bit the bullet and updated my stack to Node 0.12 (was less painful than I imagined), after some work got the staging dev up, and by stress testing I was able to reproduce the exact same error after a while.
Apparently the issue is not fixed with Node 0.12 or the latest MariaSQL module
@mscdex if you have any rewrite code, or code with extra logging you want me to test I can probably reproduce the error within a few minutes.
@gould @nicokaiser @zingaburga I just pushed support for both protocol-level pings and TCP keepalive to the rewrite branch.
The protocol-level pings can be enabled by setting pingInactive to how many milliseconds to wait before sending a ping when no queries are pending, and pingWaitRes to how many milliseconds to wait for a ping response before assuming a lost/dead connection.
The TCP keepalive functionality can be used by setting at least tcpKeepalive to the time in seconds that the socket needs to be idle until the OS starts sending keepalive probes. The other two options (if your OS supports them, otherwise they are no-ops) are tcpKeepaliveCnt which is the number of (unanswered) probes the OS should send before dropping the connection, and tcpKeepaliveIntvl which is the time in seconds between individual keepalive probes.
The TCP keepalive settings are for additional help in detecting a dead connection, especially if the connection gets severed while a query is executing on the server. The protocol-level ping would not detect that since pings can only be sent when no queries are in progress. However the protocol-level pings are important because those keep the server from closing an idle connection (they should be sent at some interval less than wait_timeout on the server).
If you're feeling adventurous, try installing from that rewrite branch (e.g. npm install https://github.com/mscdex/node-mariasql/tarball/rewrite). I should warn you that the API isn't 100% backwards compatible and there isn't new documentation (yet). A quick example though is:
var SQLConnection = require('mariasql');
var db = new SQLConnection({
host: '127.0.0.1',
port: 3306,
user: 'foo',
password: 'bar',
db: 'test'
});
// for buffered rows
db.query('SELECT "foo" as Foo, "bar" as Bar, 3 as Three', function(err, rows) {
console.dir(rows);
});
// for streamed rows
db.query('SELECT "foo" as Foo, "bar" as Bar, 3 as Three')
.on('result', function(res) {
console.log('result start');
// `res` is a streams2 object in object mode
res.on('data', function(row) {
console.log('row', row);
}).on('end', function() {
console.log('result end');
});
}).on('end', function() {
console.log('all done!');
db.end();
});
The keep-alive-features are very nice - but the main reason for this ticket was the failing assert(). If the connection drops (for any case), it should not exit() the complete node process without any chance to handle the error in a normal way.
Or is this fixed too in the rewrite-branch?
@not-implemented Right, that's a C++ assert and can't be handled by JS land. The C++ side of mariasql was simplified a lot, so I'm hoping the assertion error won't happen now. However I need other people to test also.
This error did not happen for a long time now - I'll close the ticket. Thanks.
I am still having the same issue when connecting to the database (first action) when using Debian 8 x64.
Below you can see my code for connecting to the database:
dbConnection = new DatabaseClient();
dbConnection.connect
({
host: connectionProperties.Host,
user: connectionProperties.Username,
password: connectionProperties.Password,
db: connectionProperties.Database
});
dbConnection.on("ready", function()
{
connectionEstablished = true;
console.log("Database connection established!");
if ((onFullfilled !== undefined) && (onFullfilled !== null))
{
onFullfilled();
}
})
.on("error", function(connectionError)
{
connectionEstablished = false;
if ((onRejected !== undefined) && (onRejected !== null))
{
onRejected(connectionError);
}
});
Yes, actually we get this error again. Still with the current version ... (now in line 975):
../src/binding.cc:975: static void Client::cb_poll(uv_poll_t*, int, int): Assertion `status == 0' failed.
It seems, this happens on connection-errors at connection-startup or at authentication. Before the assertion error we often get a "Authentication with backend failed. Session will be closed." error. (The message is actually from MaxScale which is in between Node and MySQL-Server here)
any progress here? I got same error:
node: ../src/binding.cc:992: static void Client::cb_poll(uv_poll_t*, int, int): Assertion `status == 0' failed.
Seeing this on Node 4.6.1 with node-mariasql version 0.2.6. Anyone seeing this with Node 6.x or later?
I have the same issue:
root@**:/opt/development/.chatserver# static void Client::cb_poll(uv_poll_t, int, int): Assertion `status == 0' failed.
Node.js: v9.11.1
mariasql: latest, today installed
I think, the problem is an connection timeout beside MariaDB. I will try to pool the connection tomorrow. Be sure, that you have close the connection after a longer time. The connection will be brokened and getting a timeout, when you let it open without transactions.
A little (bad) idea to prevent the Error:
Create an simple interval to hold the connection active. Execute an unrelated query or whatever to prevent the connection timeout.
A little (bad) idea to prevent the Error: Create an simple
intervalto hold the connection active. Execute an unrelated query or whatever to prevent the connection timeout.
Same did I, just cause I didn't find any solution.
For this moment just keep requesting ping query SELECT 1;
setting param pingInterval on connection doesn't help me.
No, this project seems to be abandoned. We switched to mysqljs over time.