Maximum call stack size exceeded while trying to import with es-import-aliases

[levineran]

I tried to import a 10.2 mb file which contains aliases export.

{
  "invoice-16603": {
    "aliases": {}
  },
  "invoice-27934": {
    "aliases": {}
  },
  "ivy-attachment": {
    "aliases": {
      "attachment-1": {
        "filter": {
          "term": {
            "company_id": 1
          }
        },
        "index_routing": "attachment-1",
        "search_routing": "attachment-1"
      },
      "attachment-10": {
        "filter": {
          "term": {
            "company_id": 10
          }
        },
        "index_routing":

...

import failed : es-import-aliases --url http://localhost:9200 --file /ivy/export.json

RangeError: Maximum call stack size exceeded
    at next (/usr/local/lib/node_modules/async/dist/async.js:918:25)
    at replenish (/usr/local/lib/node_modules/async/dist/async.js:989:28)
    at iterateeCallback (/usr/local/lib/node_modules/async/dist/async.js:983:17)
    at /usr/local/lib/node_modules/async/dist/async.js:958:16
    at /usr/local/lib/node_modules/elasticsearch-tools/lib/importer.js:75:11
    at /usr/local/lib/node_modules/async/dist/async.js:3096:16
    at replenish (/usr/local/lib/node_modules/async/dist/async.js:998:17)
    at iterateeCallback (/usr/local/lib/node_modules/async/dist/async.js:983:17)
    at /usr/local/lib/node_modules/async/dist/async.js:958:16
    at /usr/local/lib/node_modules/elasticsearch-tools/lib/importer.js:75:11

does any solution exist?

[skratchdot]

Interesting. It looks like we need to use a streaming reader for aliases, mappings, and settings. Currently, only the bulk import works w/ huge files due to the line-by-line package (https://github.com/skratchdot/elasticsearch-tools/blob/master/lib/es-import-bulk.js#L8).

Here is the line for the other imports we need to change: https://github.com/skratchdot/elasticsearch-tools/blob/master/lib/importer.js#L44