Refactor migrate / install scripts to keep up to SRP

[EliasKotlyar]

eko@eko-GE62VR-6RF:~/Schreibtisch/vue-storefront-api$ npm run migrate

> [email protected] migrate /home/eko/Schreibtisch/vue-storefront-api
> node node_modules/migrate/bin/migrate

Elasticsearch INFO: 2018-03-05T20:07:01Z
  Adding connection to http://localhost:9200/

  up : 1513603161654-reindex.js
Elasticsearch DEBUG: 2018-03-05T20:07:01Z
  starting request {
    "method": "POST",
    "path": "/_reindex",
    "body": {
      "source": {
        "index": "vue_storefront_catalog_temp"
      },
      "dest": {
        "index": "vue_storefront_catalog"
      }
    },
    "query": {}
  }
  

Elasticsearch DEBUG: 2018-03-05T20:07:01Z
  Request complete

{ Error: [index_not_found_exception] no such index, with { resource.type="index_or_alias" & resource.id="vue_storefront_catalog_temp" & index_uuid="_na_" & index="vue_storefront_catalog_temp" }
    at respond (/home/eko/Schreibtisch/vue-storefront-api/node_modules/elasticsearch/src/lib/transport.js:307:15)
    at checkRespForFailure (/home/eko/Schreibtisch/vue-storefront-api/node_modules/elasticsearch/src/lib/transport.js:266:7)
    at HttpConnector.<anonymous> (/home/eko/Schreibtisch/vue-storefront-api/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)
    at IncomingMessage.bound (/home/eko/Schreibtisch/vue-storefront-api/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1055:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
  status: 404,
  displayName: 'NotFound',
  message: '[index_not_found_exception] no such index, with { resource.type="index_or_alias" & resource.id="vue_storefront_catalog_temp" & index_uuid="_na_" & index="vue_storefront_catalog_temp" }',
  path: '/_reindex',
  query: {},
  body: 
   { error: 
      { root_cause: [Array],
        type: 'index_not_found_exception',
        reason: 'no such index',
        'resource.type': 'index_or_alias',
        'resource.id': 'vue_storefront_catalog_temp',
        index_uuid: '_na_',
        index: 'vue_storefront_catalog_temp' },
     status: 404 },
  statusCode: 404,
  response: '{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"vue_storefront_catalog_temp","index_uuid":"_na_","index":"vue_storefront_catalog_temp"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"vue_storefront_catalog_temp","index_uuid":"_na_","index":"vue_storefront_catalog_temp"},"status":404}',
  toString: [Function],
  toJSON: [Function] }
  error : [index_not_found_exception] no such index, with { resource.type="index_or_alias" & resource.id="vue_storefront_catalog_temp" & index_uuid="_na_" & index="vue_storefront_catalog_temp" } :: {"path":"/_reindex","query":{},"body":"{\"source\":{\"index\":\"vue_storefront_catalog_temp\"},\"dest\":{\"index\":\"vue_storefront_catalog\"}}","statusCode":404,"response":"{\"error\":{\"root_cause\":[{\"type\":\"index_not_found_exception\",\"reason\":\"no such index\",\"resource.type\":\"index_or_alias\",\"resource.id\":\"vue_storefront_catalog_temp\",\"index_uuid\":\"_na_\",\"index\":\"vue_storefront_catalog_temp\"}],\"type\":\"index_not_found_exception\",\"reason\":\"no such index\",\"resource.type\":\"index_or_alias\",\"resource.id\":\"vue_storefront_catalog_temp\",\"index_uuid\":\"_na_\",\"index\":\"vue_storefront_catalog_temp\"},\"status\":404}"}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] migrate: `node node_modules/m

[pkarw]

vue_storefront_catalog_temp is created by npm run restore command and then after the migration ran its deleted

So after you run the vue-storefront installer - you will be not able to re-run the migrations without running the first step - npm run restore

[daaru00]

Hi, IMHO this issue is not closed, is quite bit confusing that the migration not working if you do not run first yarn restore, migration should create all need index to start with an empty dataset.

If is behaviour is made "by design" ok, but please change the order of installation step in the README that say first run migrate and then restore.

[pkarw]

@daaru00 good point. I'll check it. I think it should be modified - you're right; I mean - migrations shouldn't be related to the restore script. You could modify it if You like.

To do so You probably need to modify the installation procedure (core/scripts/installer.js + README file) and the npm run restore to sth like this:

First variant - to use the default data set - to be used by the installer:

yarn restore
yarn db rebuild
yarn db migrate

Second variant - to use the tempty data set - to be available for the manual installation:

yarn db new
yarn db migrate

Migrate script should be always at the end as there is a possibility that for example some migrations are adding specific Elastic data entities that were not available in the dump.

[daaru00]

Hi @pkarw, cloud be a solution adding the missing index vue_storefront_catalog_temp from the migration?

I add it here: https://github.com/DivanteLtd/vue-storefront-api/blob/master/migrations/1513602693128-create_new_index.js#L8 in this way:

module.exports.up = next => {
  createIndex(common.db, config.elasticsearch.indices[0], () => {
    createIndex(common.db, config.elasticsearch.indices[0] + '_temp', next)
  })
}

then I run npm run migrate and it works without errors.

[pkarw]

We can use it as a workaround but it's not an elegant way to solve this issue :) What if the _temp already exists (after somebody ran npm run migrate?)

We can use it as a workaround

[pkarw]

In that case please just create a PR with this change :)

[WazzaJB]

Just to add a note here for anyone else battling this issue.

If you have already run the migrate command, applying the patch above will still yield the same error unless you reverse the migration.

Run yarn run migrate down, apply the patch, then re-run yarn run migrate and you should be fine 👍