Additional out of stock check leads to ghost products on catalog and search pages.

[mfickers]

After update to the newest extension version, we've noticed an issue with out of stock products being counted in filters and pagination, messing with the page sizes and creating filters that lead to empty listings.

Preconditions

Magento Version : 2.4.1

ElasticSuite Version : 2.10.3

Environment : Developer/Production

Steps to reproduce

1. All indices on "On Schedule", no cron configured
2. Set the available Qty for a product to 0 for the assigned source
3. Reload the category page without flushing any caches or running the indexers

Expected result

1. Either the product should be in the grid and the filters and pagination should not change
2. Or the product should be missing and the product count in filters and pagination should decrease by one

Actual result

1. The product will not show up in the grid
2. The filters and pagination behave like the product was there and do not change

There are five products in the category with a page size of three. Four of these products are salable and one is out of stock. My inventory_stock_3 index table marks the product as not salable. The catalogsearch_fulltext index was not updated yet.

I think that this issue might be related to other problems with filters and pagination that were reported. For example https://github.com/Smile-SA/elasticsuite/issues/1912.

When initializing the product collection for the listing, there is a select query on catalog_product_entity with all entity_id returned from elasticsearch query. Magento MSI adds another filter to this query, removing all products marked as out of stock in the stock status index table. This leads to the product missing from the items loaded for the current page, without changing the overall collection item count.

After running the catalogsearch_fulltext indexer the problem disappears.

SELECT e.sku FROM catalog_product_flat_1 AS e INNER JOIN catalog_product_index_price AS price_index ON price_index.entity_id = e.entity_id AND price_index.customer_group_id = 0 AND price_index.website_id = '1' LEFT JOIN review_entity_summary AS review_summary ON e.entity_id = review_summary.entity_pk_value AND review_summary.store_id = 1 AND review_summary.entity_type = (SELECT `review_entity`.`entity_id` FROM `review_entity` WHERE (entity_code = 'product')) INNER JOIN catalog_product_entity AS product ON product.entity_id = e.entity_id INNER JOIN inventory_stock_3 AS stock_status_index ON product.sku = stock_status_index.sku WHERE (stock_status_index.is_salable = 1) AND (e.entity_id IN (935616, 935614, 935651));

This is the query that fetches the products from the product entity table. I set page size to three, but the query only returns 2 products of the three entity ids that are given by elasticsearch query. When I remove the where clause for stock_status_index, all three products are returned. This where clause is added in \Magento\CatalogInventory\Model\AddStockStatusToCollection this plugin.

The underlying problem is the elasticsearch index getting out of date, but the plugin is not needed as the stock status is already tracked in the fulltext index. So in my opinion this plugin should be disabled by this extension.

[Quazz]

I believe that was actually a core Magento bug.

Magento 2.4.2 changelog contains this:

Partial re-indexing of large catalogs now works as expected. Previously, products randomly disappeared from the storefront category page during re-indexing with the catalogsearch_fulltext partial indexer.

[mfickers]

Thank you for that piece of info. This explains why the two indices where getting out of sync. We'll get to that update as soon as possible and see how that improves the situation.

However, there is still the issue of duplicate out of stock checks for the product collection. As long as both indices contain the same information about salability this won't matter, but as you can see above this can lead to weird behavior and should not be needed.

[gunwunbun]

This is still a problem with ElasicSuite 2.10.0 and Magento 2.4.2. I don't have the time to fully figure out why it's happening but for now the cause seems to be AddStockStatusToCollection. Fully removing the plugin in vendor/magento/module-catalog-inventory/etc/frontend/di.xml fixes it for me.