"Fast-Headless-Import and store minimal product data option" too many requests

[Nick-BC]

Expected behavior

Reported by https://www.scotlightdirect.co.uk/

We are currently having an issue with changing our wordpress sync option to the new "Fast-Headless-Import and store minimal product data option". When we select this BC send too many request the site slowes down massively and it crashes. There seems to be an issue with web hooks

Webhooks appear to be blocked and in retry after enabling the setting - https://developer.bigcommerce.com/docs/9a6037ffe79fd-webhooks-overview#callback-retry-mechanism

See thread - https://bigcommerce.slack.com/archives/C7XEZCSBF/p1656241083030689

[jamescscotlightdirect]

Hello,

Thank you for the most recent update the website no longer crashes on fast headless mode however I am still experiencing errors. Should I raise this with a BC support ticket or is it in the same scope? [2022-08-08 09:01:46] BigCommerce.DEBUG: Importing terms for bigcommerce_category taxonomy {"page":114,"limit":2,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:46] BigCommerce.DEBUG: bigcommerce_category import ready for next page of terms {"next":115,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:49] BigCommerce.INFO: Running import task {"state":"updating_categories","description":"Updating Categories"} [] [2022-08-08 09:01:49] BigCommerce.DEBUG: Importing terms for bigcommerce_category taxonomy {"page":115,"limit":2,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:49] BigCommerce.DEBUG: bigcommerce_category import ready for next page of terms {"next":116,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:51] BigCommerce.INFO: Running import task {"state":"updating_categories","description":"Updating Categories"} [] [2022-08-08 09:01:51] BigCommerce.DEBUG: Importing terms for bigcommerce_category taxonomy {"page":116,"limit":2,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:51] BigCommerce.DEBUG: bigcommerce_category import ready for next page of terms {"next":117,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:53] BigCommerce.INFO: Running import task {"state":"updating_categories","description":"Updating Categories"} [] [2022-08-08 09:01:53] BigCommerce.DEBUG: Importing terms for bigcommerce_category taxonomy {"page":117,"limit":2,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:53] BigCommerce.DEBUG: bigcommerce_category import ready for next page of terms {"next":118,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:55] BigCommerce.INFO: Running import task {"state":"updating_categories","description":"Updating Categories"} [] [2022-08-08 09:01:55] BigCommerce.DEBUG: Importing terms for bigcommerce_category taxonomy {"page":118,"limit":2,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:55] BigCommerce.DEBUG: bigcommerce_category import ready for next page of terms {"next":119,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:57] BigCommerce.INFO: Running import task {"state":"updating_categories","description":"Updating Categories"} [] [2022-08-08 09:01:57] BigCommerce.DEBUG: Importing terms for bigcommerce_category taxonomy {"page":119,"limit":2,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:57] BigCommerce.DEBUG: bigcommerce_category import ready for next page of terms {"next":120,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:01:59] BigCommerce.INFO: Running import task {"state":"updating_categories","description":"Updating Categories"} [] [2022-08-08 09:01:59] BigCommerce.DEBUG: Importing terms for bigcommerce_category taxonomy {"page":120,"limit":2,"taxonomy":"bigcommerce_category"} [] [2022-08-08 09:02:00] BigCommerce.INFO: Running import task {"state":"updated_categories","description":"Updating Brands"} [] [2022-08-08 09:02:00] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":1,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:00] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:03] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:03] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":2,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:03] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":3,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:05] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:05] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":3,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:05] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":4,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:07] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:07] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":4,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:07] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":5,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:08] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:08] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":5,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:09] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":6,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:11] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:11] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":6,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:12] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":7,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:14] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:14] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":7,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:14] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":8,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:16] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:16] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":8,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:17] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":9,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:19] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:19] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":9,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:19] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":10,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:21] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:21] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":10,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:21] BigCommerce.DEBUG: bigcommerce_brand import ready for next page of terms {"next":11,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:22] BigCommerce.INFO: Running import task {"state":"updating_brands","description":"Updating Brands"} [] [2022-08-08 09:02:23] BigCommerce.DEBUG: Importing terms for bigcommerce_brand taxonomy {"page":11,"limit":2,"taxonomy":"bigcommerce_brand"} [] [2022-08-08 09:02:25] BigCommerce.INFO: Running import task {"state":"updated_brands","description":"Fetching products data from the BigCommerce API for channel Scotlight Direct WordPress"} [] [2022-08-08 09:02:25] BigCommerce.DEBUG: Array [] [] [2022-08-08 09:02:25] BigCommerce.DEBUG: Array [] [] [2022-08-08 09:02:25] BigCommerce.DEBUG: Array [] [] [2022-08-08 09:02:26] BigCommerce.ERROR: GraphQL credentials were missing. No token was sent. [] [] [2022-08-08 09:02:26] BigCommerce.DEBUG: #0 /home/customer/www/scotlightdirect.co.uk/public_html/wp-content/plugins/bigcommerce/src/BigCommerce/GraphQL/BaseGQL.php(146): BigCommerce\GraphQL\BaseGQL->parse_response(Array) #1 /home/customer/www/scotlightdirect.co.uk/public_html/wp-content/plugins/bigcommerce/src/BigCommerce/GraphQL/GraphQL_Processor.php(58): BigCommerce\GraphQL\BaseGQL->make_request('{"query":"query...') #2 /home/customer/www/scotlightdirect.co.uk/public_html/wp-content/plugins/bigcommerce/src/BigCommerce/Import/Processors/Headless_Product_Processor.php(71): BigCommerce\GraphQL\GraphQL_Processor->request_paginated_products(50) #3 /home/customer/www/scotlightdirect.co.uk/public_html/wp-content/plugins/bigcommerce/src/BigCommerce/Container/Import.php(304): BigCommerce\Import\Processors\Headless_Product_Processor->run() #4 /home/customer/www/scotlightdirect.co.uk/public_html/wp-content/plugins/bigcommerce/src/BigCommerce/Import/Task_Manager.php(97): BigCommerce\Container\Import->BigCommerce\Container{closure}('updated_brands') #5 /home/customer/www/scotlightdirect.co.uk/public_html/wp-content/plugins/bigcommerce/src/BigCommerce/Container/Import.php(340): BigCommerce\Import\Task_Manager->run_next('updated_brands') #6 /home/customer/www/scotlightdirect.co.uk/public_html/wp-includes/class-wp-hook.php(307): BigCommerce\Container\Import->BigCommerce\Container{closure}('updated_brands') #7 /home/customer/www/scotlightdirect.co.uk/public_html/wp-includes/class-wp-hook.php(331): WP_Hook->apply_filters(NULL, Array) #8 /home/customer/www/scotlightdirect.co.uk/public_html/wp-includes/plugin.php(476): WP_Hook->do_action(Array) #9 /home/customer/www/scotlightdirect.co.uk/public_html/wp-content/plugins/bigcommerce/src/BigCommerce/Import/Runner/Cron_Runner.php(51): do_action('bigcommerce/imp...', 'updated_brands') #10 /home/customer/www/scotlightdirect.co.uk/public_html/wp-content/plugins/bigcommerce/src/BigCommerce/Container/Import.php(115): BigCommerce\Import\Runner\Cron_Runner->continue_import() #11 /home/customer/www/scotlightdirect.co.uk/public_html/wp-includes/class-wp-hook.php(305): BigCommerce\Container\Import->BigCommerce\Container{closure}() #12 /home/customer/www/scotlightdirect.co.uk/public_html/wp-includes/class-wp-hook.php(331): WP_Hook->apply_filters('', Array) #13 /home/customer/www/scotlightdirect.co.uk/public_html/wp-includes/plugin.php(524): WP_Hook->do_action(Array) #14 /home/customer/www/scotlightdirect.co.uk/public_html/wp-cron.php(138): do_action_ref_array('bigcommerce_con...', Array) #15 {main} [] []

[yuenvision]

@jamescscotlightdirect The second issue doesn't appear to be webhook-related, so it's out of scope of this specific github issue. We'll need a separate issue opened for that, along with replication steps.

[jamescscotlightdirect]

@yuenvision This can now be closed thank you