esbulk icon indicating copy to clipboard operation
esbulk copied to clipboard

Published 20 hours ago • verified publisher icon miku

Workers

Open claytondukes opened this issue 5 years ago • 1 comments

I'm curious about the -w option. I have a 40GB file to import but it doesn't seem to matter how much CPU or mem I give the system. I have 192 cores and ~300GB ram but can't get -w to go past -w 7. I have 6 nodes in this cluster.

2020/06/22 16:20:36 error during bulk operation, check error details; maybe try fewer workers (-w) or increase thread_pool.bulk.queue_size in your nodes

FWIW, htop shows barely any utilization while I am running the import. Am I doing something wrong?

claytondukes avatar Jun 22 '20 20:06 claytondukes

Thanks for the bug report! I'll prioritize the improved error reporting, which I meant to implement for some time now.

Apart from that, I do not see any error (and usage should be more or less self-explanatory).

miku avatar Jun 22 '20 21:06 miku