loopback-connector-elastic-search icon indicating copy to clipboard operation
loopback-connector-elastic-search copied to clipboard

How to run boot scripts when Es connector is not initialised

Open aquid opened this issue 7 years ago • 4 comments

If we try to run a boot script now without our connector being initialised we get back an error saying Index doesn't exist.

here's the error message

Unhandled rejection [index_not_found_exception] no such index, with { resource.type=index_or_alias resource.id=blubox-elastic index=blubox-elastic } :: {"path":"/blubox-elastic/Bluboxer/_search","query":{"size":1,"from":0},"body":"{\"sort\":[\"_uid\"],\"query\":{\"bool\":{\"must\":[{\"match\":{\"email\":\"[email protected]\"}}]}}}","statusCode":404,"response":"{\"error\":{\"root_cause\":[{\"type\":\"index_not_found_exception\",\"reason\":\"no such index\",\"resource.type\":\"index_or_alias\",\"resource.id\":\"blubox-elastic\",\"index\":\"blubox-elastic\"}],\"type\":\"index_not_found_exception\",\"reason\":\"no such index\",\"resource.type\":\"index_or_alias\",\"resource.id\":\"blubox-elastic\",\"index\":\"blubox-elastic\"},\"status\":404}"}
  at respond (/Users/shahwarcoder/Blubox/node_modules/elasticsearch/src/lib/transport.js:289:15)
  at checkRespForFailure (/Users/shahwarcoder/Blubox/node_modules/elasticsearch/src/lib/transport.js:248:7)
  at HttpConnector.<anonymous> (/Users/shahwarcoder/Blubox/node_modules/elasticsearch/src/lib/connectors/http.js:164:7)
  at IncomingMessage.wrapper (/Users/shahwarcoder/Blubox/node_modules/lodash/index.js:3095:19)
  at emitNone (events.js:72:20)
  at IncomingMessage.emit (events.js:166:7)
  at endReadableNT (_stream_readable.js:913:12)
  at nextTickCallbackWith2Args (node.js:442:9)
  at process._tickDomainCallback (node.js:397:17)
  at process.fallback (/Users/shahwarcoder/Blubox/node_modules/async-listener/index.js:482:15)

cc @pulkitsinghal

aquid avatar Jan 09 '17 21:01 aquid

@aquid - we received the following answer in the chat room:

Heath Morrison @doublemarked Jan 10 16:16
@pulkitsinghal this should not be necessary
Datasource initialization occurs before boot scripts are executed.
The datasource initialize method has a callback,
which should not be called until the datasource is fully initialized.
(afaik)
I see in your connector that you are indeed calling this callback only on connect, which makes me wonder what @aquid means when he says "without our connector being initialised” in strongloop-community/loopback-connector-elastic-search#66

pulkitsinghal avatar Jan 17 '17 13:01 pulkitsinghal

@aquid I use the es connector like so as a boot script: (almost on first position, since it starts with "e" ;)):


module.exports = function (app, cb) {
  console.info('Boot waiting for ElasticSearch initialization...');
  var connected = false;

  // As soon as the datasource has been connected, move on.
  app.datasources.db.once('connected', function () {
    console.info('ElasticSearch connected, boot continuing');
    connected = true;
    cb();
  });
  
  // I need this, otherwise, if no ES is available, it will hang forever. PM2 then autorestarts.
  setTimeout(function(){
    if(!connected) {
      cb(new Error("retrying elasticsearch connection"));
    }
  }, 5000);
};```

Only "problem", the es connector does NOT retry infinitely, so at some point, the application just hangs, that's why I "restart" by throwing an error and pm2 takes over.

wolfgang-s avatar Feb 25 '17 06:02 wolfgang-s

@wolfgang-s So the above piece of code goes before each boot scripts or just one boot scripts which run before all other and checks if elastic is up and running.

Can't we do a refresh configuration for index too? Is it possible?

aquid avatar Feb 25 '17 14:02 aquid

This is a normal loopback boot module. Runs once before the rest of the boot modules.

I'm using PM2 which restarts the node app on failure (when it takes more than 5 seconds).

Actually almost the same as here: https://github.com/strongloop-community/loopback-connector-elastic-search/issues/7

wolfgang-s avatar Feb 25 '17 17:02 wolfgang-s