elasticsuite icon indicating copy to clipboard operation
elasticsuite copied to clipboard

Allow configuring a specific number of primary shards per index type

Open rbayet opened this issue 1 year ago • 1 comments

Is your feature request related to a problem? Please describe. Adobe Cloud pushes for using by default 3 primary shards (and 2 replicas, but that's ok) on their 3 nodes cluster architecture whatever the context (aka the size of the catalog product index). Even if Elasticsuite integrates now a warning that, depending on the size of the catalog_product Elasticsearch index, that number of primary shards might not be warranted, not all users will apply the change back to 1 primary shard.

The main issue is that number of 3 primary shards is then applied to all other Elasticsuite created indices : catalog_category, thesaurus, behavioral tracking data, cms related, etc... This is not a dramatic issue per se on a Adobe Commerce environment with a single store, but

  • when the Elasticsearch cluster is shared between "staging" or "integration" environments
  • or when the Magento install is a (many) multiple store setups it can lead to the breaking of the rule of "1GB of Heap memory per 20 shards per node".

And Adobe Cloud hosting will usually recommend to the user to disable features like the tracking.

Describe the solution you'd like A system of either static or dynamic Stores/Configuration base settings for Elasticsuite to allow specifying the primary shards to use for other kind of indices besides catalog_product. The idea being :

  • if there is no specific setting for a given index type, then use the default and existing system.xml configuration variable for number of primary shards
  • otherwise, use that specific setting

By default, support specific settings for the index types included in Elasticsuite Open Source

  • catalog category
  • thesaurus
  • tracking log event
  • tracking log session

rbayet avatar Sep 28 '23 07:09 rbayet

You can take some inspirations from this Source Model which is fetching informations from XML :

https://github.com/Smile-SA/elasticsuite/blob/2.11.x/src/module-elasticsuite-core/Model/Search/Request/Source/Containers.php

In your particular Source Model, you should probably use this class to get the indices configuration :

https://github.com/Smile-SA/elasticsuite/blob/2.11.x/src/module-elasticsuite-core/Index/Indices/Config.php

romainruaud avatar Dec 11 '23 14:12 romainruaud

This has been done, I close.

romainruaud avatar Apr 29 '24 13:04 romainruaud