elasticsearch-stress-test icon indicating copy to clipboard operation
elasticsearch-stress-test copied to clipboard

Published 20 hours ago • verified publisher icon logzio

Not able to generate high load

Open orzilca opened this issue 5 years ago • 5 comments

Hi, great tool!

Having said that, there are some issues to be solved :)

  • it seems like its impossible to generate heavy load. no matter which setting I try (clients, duration, doc size), the maximum load of docs/second I can reach is 2000. i have tried setting the clients count to big numbers such as 5000, but I dont see indexing rate changes on Marvel (its never passes to 2000 docs/s). maybe I am missing something?
  • Missing testing of non-bulk indexing. (es.index(...))

orzilca avatar Jul 22 '18 16:07 orzilca

Actually it seems like I'm not able generate load of more than 100 docs/s. It shows 2000 because I use 1 replica.

orzilca avatar Jul 22 '18 16:07 orzilca

Wierd, i was always able to generate load with this. But this tool was written a long time ago, and frankly, for real load testing we are using this tool which we also wrote - https://github.com/logzio/elasticsearch-benchmark-tool As it is a lot more comprehensive and can generate more "real-life" scenarios

roiravhon avatar Jul 23 '18 09:07 roiravhon

Which version of ES are you testing it against?

roiravhon avatar Jul 23 '18 09:07 roiravhon

6.3.1

I'll take a look at the new version. There's also Rally by elastic which seems like a good benchmarking tool.

orzilca avatar Jul 23 '18 10:07 orzilca

Yep, also a good tool. It was just too comprehensive and complicated for our benchmark needs. This tool was written for ES 2.x, so might need some adjustments for 6.x needs :)

roiravhon avatar Jul 24 '18 07:07 roiravhon