mosca icon indicating copy to clipboard operation
mosca copied to clipboard

Mosca Crashing Server

Open ccravens opened this issue 8 years ago • 9 comments

It appears that having the Mosca library included in the server may be crashing it. At, what appears to be random intervals, we get the following:

/Users/ccravens/Business/company/Clients/project/src/project-core/node_modules/ascoltatori/node_modules/mongodb/lib/utils.js:98
    process.nextTick(function() { throw err; });
                                  ^

Error: Cannot stat if collection is capped or not
    at /Users/ccravens/Business/company/Clients/project/src/project-core/node_modules/ascoltatori/lib/mongo_ascoltatore.js:235:26
    at handleCallback (/Users/ccravens/Business/company/Clients/project/src/project-core/node_modules/ascoltatori/node_modules/mongodb/lib/utils.js:96:12)
    at /Users/ccravens/Business/company/Clients/project/src/project-core/node_modules/ascoltatori/node_modules/mongodb/lib/collection.js:1503:20

ccravens avatar Jul 21 '16 17:07 ccravens

Actually this might be due to the connection to the database being poor. Might not be an ascoltatori issue at all... Just wanted to verify everyone's thoughts before closing it.

ccravens avatar Jul 21 '16 18:07 ccravens

I think it's a combination of both. The connection is poor, and ascoltatori is failing too early: https://github.com/mcollina/ascoltatori/blob/master/lib/mongo_ascoltatore.js#L232-L237.

I think we should add a retry mechanism there.

mcollina avatar Jul 22 '16 06:07 mcollina

@mcollina has there been any fix for this? can you suggest a workaround? My MQTT just stops working after this error :(

dushyantbangal avatar Aug 17 '17 06:08 dushyantbangal

No I have not worked on a fix. Would you like to send a PR?

mcollina avatar Aug 17 '17 14:08 mcollina

I've come up with the following (dirty) retry logic:

if (err) 
{
    if(!retryCount)
        retryCount=0;
    if(retryCount<that._maxRetry)
    {
        debug('checkCappedAndPoll -> Cannot stat isCapped. Retry');
        
        setTimeout(function(){
            that._checkCappedAndPoll(latest, ++retryCount);
        },500)
            
        return;
    }else{
        debug('checkCappedAndPoll -> Cannot stat isCapped. Give up');

        that.emit('error', new Error('Cannot stat if collection is capped or not'));
        that._handlingCursorFailure = false;
        return;
    }
}

Should I be doing it like this? Should I be putting the setTimeout of 500ms or directly retrying or putting larger timeout? Should I be using something else than your that._maxRetry ?

dushyantbangal avatar Aug 18 '17 08:08 dushyantbangal

@dushyantbangal does that solves your problem?

I think we can increase the maximum retry count to 5 and do something like setTimeout(func, 100 * retryCount), so that it increase overtime (and the first one is 0).

mcollina avatar Aug 18 '17 08:08 mcollina

I'm not really able to recreate the issue, it happens sometimes. But I did try the following:

  1. Set error=true for count 0,1,2
  2. Execute the broker with DEBUG=ascoltatori:mongodb

For counts 0,1,2 the execution went in retry, and after count 2 the polls were successful.

dushyantbangal avatar Aug 18 '17 09:08 dushyantbangal

Can you send a PR then?

mcollina avatar Aug 18 '17 09:08 mcollina

Sure, will do!

dushyantbangal avatar Aug 18 '17 09:08 dushyantbangal