Products belonging to multiple bundles become increasingly slower to save

[craig-bartlett]

Hi,

I'm currently investigating an issue with saving simple products that belong to multiple large bundles, which I've replicated on a clean install. If a product belongs to many large bundles (in our case one product belongs to 80 different bundles with a many as 5 options with 40 products in each) and these products are assigned to stocks/sources, it can take several minutes to save that product even when no changes have been made to it. I believe this is down to the shipping validator which checks to make sure that every child product of every related parent bundle either has the same source set or is set to ship separately, but has to load them all first. I've included some notes on this at the end.

Preconditions (*)

1. Magento 2.4.X. (I've tested and replicated on 2.4.1-p1 and 2.4.2)
2. Multiple sources assigned to a non-default stock
3. Multiple large bundles with at least one common child product, assigned to various sources
4. Flat catalog is left disabled

Steps to reproduce (*)

1. Generate dummy data for a large store using the performance toolkit. I modified the "larger" profile and added a thousand large bundles. The number of users and orders is irrelevant, as are the number of simple products as the bundles generate their own. The only relevant changes I made here are as follows:

        <websites>5</websites> <!-- Number of websites to generate -->
        <store_groups>5</store_groups> <!--Number of stores-->
        <store_views>5</store_views> <!-- Number of store views -->
        <assign_entities_to_all_websites>1</assign_entities_to_all_websites> <!-- Whether to assign all products per each website -->
        <simple_products>300</simple_products> <!-- Simple products count -->
	<bundle_products>1000</bundle_products>
	<!-- Number of options per each product -->
	<bundle_products_options>4</bundle_products_options>
	<!-- Number of simple products per each option -->
	<bundle_products_variation>30</bundle_products_variation>

2. Create some sources (10 in my case) and assign them to a new stock
3. Assign all products to the new sources
4. As the generator creates unique products for each bundle I then had to assign one product as an option for multiple bundles to replicate our live server. I did this to 80 bundles by modifying the catalog_product_bundle_selection table and changing 80 different bundle options to have the product_id of my product.
5. I ran a full reindex but I don't think it made a difference, noting just in case
6. Now select the product in Catalog > Products and without making changes, hit save

Expected result (*)

1. Product should save without delay

Actual result (*)

1. Product takes several minutes to save

Findings

I've run through this process with Xdebug in an attempt to provide as much info as possible on where the slow down occurs. (For whatever reason the built-in profiler didn't record any of this so I do not have profiler output to share. I'll take a look at this again and provide it if I'm able)

• When a product is saved an observer (with event catalog_product_save_after) is triggered to save the product's stock_data property to a Stock Item object using the StockRegistry model (SaveInventoryObserver)
• StockRegistry updates the stock item and saves it using StockItemRepository
• StockItemRepository sets various properties and then calls save which triggers a plugin UpdateSourceItemAtLegacyStockItemSavePlugin. This seems to be updating source items for both legacy and non-legacy stock items
• The default source item is updated or created if needed, then passed on in UpdateSourceItemBasedOnLegacyStockItem
• SourceItemSaveWithoutLegacySynchronization just passes this on to SourceItemsSaveHandler
• This leads to the source item being validated in SourceItemsSaveHandler
• SourceItemsValidator passes this on to SourceItemValidatorChain
• Several validators are run against the source item, the slow one being ShipmentTypeValidator and occurring on line 84 where bundleProductsByChildSku is used to load all bundles the product is a child of along with the options in ProductRepository. The call to addExtensionAttributes in particular takes a long time.
• Once the products are loaded in ShipmentTypeValidator, it just checks the source codes of the bundle products if they haven't been set to ship separately to "prevent add source to product if one is part of bundle product with shipment type 'ship together'"

I'm not entirely sure how best to remedy this. In our case, all bundle products are set to ship separately so the last validator seems like it could be skipped entirely and thus bypass the slower parts. I also wondered if all products from all related bundles really have to be loaded and compared when sources haven't event been changed? It's also entirely possible I'm being an idiot and have missed something?

[m2-assistant[bot]]

Hi @craig-bartlett. Thank you for your report. To help us process this issue please make sure that you provided sufficient information.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[heerdt]

Hey @craig-bartlett !

I have the same exact issue, with an extra problem that I am using GraphQL to fetch my products and display on my storefront. Every time I save a simple product that is part of many bundles, it removes the bundles from the search while it's saving/reindexing, which can take several minutes depending on how many bundle products it's assigned to. I think it's because Magento is trying to update the stock status of each bundle.

[craig-bartlett]

Hi @heerdt

Thanks for the confirmation!

As a temporary solution I was able to get our saving time from 8+ minutes (and sometimes timing out) to about 5-10 seconds. As all of our bundles are set to ship separately I simply bypassed the time consuming validation check in ShipmentTypeValidator.php with a plugin. As mentioned above, it's line 84 (where it loads all related bundles) that's the bottle neck. As it's just checking that there no issues with bundles being set to ship together whilst the child products belong in multiple sources or with some only available in different sources, and I know that all of our bundles only ship separately, it's just wasting a huge amount of time. Obviously you don't want to do this if you ship some bundles together. Have you set your indexers to run on schedule? That should prevent them running when you save and free up some time.

I'm sure there's a better way to validate this but as of yet I've not had the chance to take a deeper look at it.