daybed icon indicating copy to clipboard operation
daybed copied to clipboard

ElasticSearch bottleneck for batch record creation

Open leplatrem opened this issue 11 years ago • 1 comments

Importing a model with 3000 inline records is 20x slower with ES enabled :|

➜  time node import.js                     
node import.js  0,19s user 0,05s system 12% cpu 1,841 total
➜  import-geojson  time node import.js
node import.js  0,17s user 0,05s system 0% cpu 37,006 total

See https://github.com/spiral-project/daybed.js/pull/28

See refresh param https://github.com/spiral-project/daybed/blob/1.0.1/daybed/indexer.py#L117

See #213

leplatrem avatar Nov 09 '14 15:11 leplatrem

Without index refresh, it's only 10x slower :) (13secs)

I guess some time should be invested in handling events asynchronously instead...

leplatrem avatar Nov 10 '14 14:11 leplatrem