elasticsearch icon indicating copy to clipboard operation
elasticsearch copied to clipboard

[ML] Increase default queue_capacity to 10_000 and decrease max queue_capacity to 100_000

Open maxhniebergall opened this issue 4 months ago • 2 comments

The default queue_capacity of 1024 has been affecting some customers who experience errors during reindex or when using ingest pipelines. Now that we have automatic chunking, this issue is more common. To prevent customers from running into this issue, we are increasing the default queue_capacity to 10'000. At the same time, we have found that the previous maximum queue_capacity of 1MM was too large, and would cause crashes if filled. We are decreasing it to what we believe is a more realistic value.

The queue_capacity of existing deployments should be unaffected by this change.

maxhniebergall avatar Oct 17 '24 18:10 maxhniebergall