wp-rocket icon indicating copy to clipboard operation
wp-rocket copied to clipboard

Published 20 hours ago • verified publisher icon wp-media

Refactor the preload

Open CrochetFeve0251 opened this issue 2 years ago • 2 comments

Description

Refactor the preload using Action Scheduler.

Fixes #(issue number)

Type of change

Please delete options that are not relevant.

  • [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • [ ] Automated Tests
  • [ ] Manual tests in local

Checklist:

Please delete the options that are not relevant.

  • [ ] My code follows the style guidelines of this project
  • [ ] I have performed a self-review of my own code
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] My changes generate no new warnings
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • [ ] New and existing unit tests pass locally with my changes
  • [ ] Any dependent changes have been merged and published in downstream modules

CrochetFeve0251 avatar Jun 13 '22 08:06 CrochetFeve0251

@CrochetFeve0251 @Tabrisrp Please find the exploratory testing notes so far below:

  • [x] 1. We need to consider a task to change the text under preload cache (please check with PO)

When you enable preloading WP Rocket will generate the cache starting with the links on your homepage followed by the sitemaps you specify. Preloading is automatically triggered when you add or update content and can also be manually triggered from the admin bar or from the WP Rocket Dashboard.

  • [x] 2. Preload transients are not deleted after deleting WPR

  • [x] 3. Sometimes Preload msg is not displayed after clicking clear and preload from the admin bar

  • [ ] 4. We only have 10 URLs in the database after preload is enabled, around 7 folders in cache => default preload on new labs generated 100+ folders => for commit d95af49 currently on newlabs only home page is preloaded for some reason

  • [x] 5. Preload is not working after fresh install although it is already enabled (nothing added to cache)

  • [x] 6. Fatal error is there when update/rollback while preload is enabled and without deactivate plugin while switching branches, will be great if we can guard this (was working fine if we deactivated plugin, however, we need recheck this using zip and upgrade/downgrade from UI)

[05-Jul-2022 14:53:27 UTC] PHP Fatal error:  Uncaught TypeError: Argument 1 passed to WP_Rocket\Event_Management\Event_Manager::add_subscriber() must be an instance of WP_Rocket\Event_Management\Subscriber_Interface, string given, called in /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Plugin.php on line 115 and defined in /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/classes/event-management/class-event-manager.php:33
Stack trace:
#0 /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Plugin.php(115): WP_Rocket\Event_Management\Event_Manager->add_subscriber()
#1 /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/main.php(50): WP_Rocket\Plugin->load()
#2 /var/www/new.rocketlabsqa.ovh/htdocs/wp-includes/class-wp-hook.php(307): rocket_init()
#3 /var/www/new.rocketlabsqa.ovh/htdocs/wp-includes/class-wp-hook.php(331): WP_Hook->apply_filters()
#4 /var/www/new.rocketlabsqa.ovh/htdocs/wp-includes/plugin.php(476): WP_Hook->do_action()
#5 /var/www/new.rocketlabsqa.ovh/htdocs/wp- in /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/classes/event-management/class-event-manager.php on line 33

While upgrading in the same scenario, we can have this error too

[05-Jul-2022 15:01:37 UTC] WordPress database error Table 'new_rocketlabsqa_ovh_AzP4U3TX.wp_wpr_rocket_cache' doesn't exist for query SELECT wpr_cache.id FROM wp_wpr_rocket_cache wpr_cache WHERE wpr_cache.url = 'https://new.rocketlabsqa.ovh/ea-page' ORDER BY wpr_cache.id DESC LIMIT 100 made by shutdown_action_hook, do_action('shutdown'), WP_Hook->do_action, WP_Hook->apply_filters, wp_ob_end_flush_all, ob_end_flush, WP_Rocket\Engine\Optimization\Buffer\Optimization->maybe_process_buffer, do_action('rocket_after_process_buffer'), WP_Hook->do_action, WP_Hook->apply_filters, WP_Rocket\Engine\Preload\Subscriber->update_cache_row, WP_Rocket\Engine\Preload\Database\Queries\Cache->create_or_update, WP_Rocket\Dependencies\Database\Query->query, WP_Rocket\Dependencies\Database\Query->get_items, WP_Rocket\Dependencies\Database\Query->get_item_ids

  • [x] 7. Preload is not working in the following scenario
1- preload is enabled and done => all URLs completed
2- deactivate preload
3- clear cache
4- enable preload => it should start adding pages to cache but it won't
  • [x] 8. Currently, if we roll back to the trunk, we aren't clearing preload from the database. Better to clear it or what do you think?
  • [x] 9. A fatal error will block a user from using the dashboard, if AS actions table was deleted then we enabled preload => recommend handling it the same way as in RUCSS.
[06-Jul-2022 11:57:22 UTC] PHP Fatal error:  Uncaught RuntimeException: Error saving action: Table 'new_rocketlabsqa_ovh_AzP4U3TX.wp_actionscheduler_actions' doesn't exist in /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Dependencies/ActionScheduler/classes/data-stores/ActionScheduler_DBStore.php:86
Stack trace:
#0 /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Dependencies/ActionScheduler/classes/ActionScheduler_ActionFactory.php(177): ActionScheduler_DBStore->save_action()
#1 /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Dependencies/ActionScheduler/classes/ActionScheduler_ActionFactory.php(69): ActionScheduler_ActionFactory->store()
#2 /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Dependencies/ActionScheduler/functions.php(19): ActionScheduler_ActionFactory->async()
#3 /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Engine/Common/Queue/AbstractASQueue.php(25): as_enqueue_async_action()
#4 /var/www/new.rocketlabsqa.ovh/htdocs/wp-content in /var/www/new.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Dependencies/ActionScheduler/classes/data-stores/ActionScheduler_DBStore.php on line 86
[06-Jul-2022 11:57:22 UTC] WordPress database error Table 'new_rocketlabsqa_ovh_AzP4U3TX.wp_actionscheduler_actions' doesn't exist for query SELECT a.action_id FROM wp_actionscheduler_actions a WHERE 1=1 AND a.status IN ('complete') AND a.last_attempt_gmt <= '2022-06-05 11:57:22' LIMIT 0, 20 made by do_action_ref_array('action_scheduler_run_queue'), WP_Hook->do_action, WP_Hook->apply_filters, ActionScheduler_QueueRunner->run, ActionScheduler_Abstract_QueueRunner->run_cleanup, ActionScheduler_QueueCleaner->clean, ActionScheduler_QueueCleaner->delete_old_actions, ActionScheduler_DBStore->query_actions
  • [x] 10. rocket_preload_clean_rows_time_event is not deleted after deleting the plugin (same for RUCSS so we shall handle it for both)

Screenshot from 2022-07-06 19-56-38

  • [x] 11. If feeds cache helper plugin is enabled, we are not adding URL/feed to cache table thus it won't be considered in next preload => better add it to the table in this case
  • [x] 12. This fatal error appeared once while exploring the branch, will be great if we can guard against it
[07-Jul-2022 10:36:57 UTC] PHP Fatal error:  Uncaught Error: Call to a member function maybe_dispatch() on null in /var/www/newer.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Engine/Common/Queue/PreloadQueueRunner.php:128
Stack trace:
#0 /var/www/newer.rocketlabsqa.ovh/htdocs/wp-includes/class-wp-hook.php(307): WP_Rocket\Engine\Common\Queue\PreloadQueueRunner->maybe_dispatch_async_request('')
#1 /var/www/newer.rocketlabsqa.ovh/htdocs/wp-includes/class-wp-hook.php(331): WP_Hook->apply_filters(NULL, Array)
#2 /var/www/newer.rocketlabsqa.ovh/htdocs/wp-includes/plugin.php(476): WP_Hook->do_action(Array)
#3 /var/www/newer.rocketlabsqa.ovh/htdocs/wp-includes/load.php(1102): do_action('shutdown')
#4 [internal function]: shutdown_action_hook()
#5 {main}
  thrown in /var/www/newer.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Engine/Common/Queue/PreloadQueueRunner.php on line 128
  • [x] 13. We need to clear the completed rocket_preload_job_preload_url from the actions table as we did with RUCSS and be controllable using a filter to avoid the actions table will grow big
  • [ ] 14. We aren't recreating the sitemap automatically if for some reason caching table was manually cleared. as per @piotrbak We should fetch sitemap at least from time to time => note: in this case even if we deleted the plugin then installed it again and re-enable preload, sitemap won't be automatically created so probably something is missing in clean up may be affecting here
  • [x] 15. We need to add a action after a page was preload with the url from the page, the fact and it is a mobile or desktop page.
  • [x] 16. A PHP fatal error occurs when using the latest ( >2.0.1( version of Composer with the feature/new-preload branch.
[15-Jul-2022 07:53:05 UTC] PHP Fatal error:  Uncaught TypeError: WP_Rocket\Event_Management\Event_Manager::add_subscriber(): Argument #1 ($subscriber) must be of type WP_Rocket\Event_Management\Subscriber_Interface, string given, called in /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Plugin.php on line 143 and defined in /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/classes/event-management/class-event-manager.php:33
Stack trace:
#0 /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/Plugin.php(143): WP_Rocket\Event_Management\Event_Manager->add_subscriber()
#1 /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-content/plugins/wp-rocket/inc/main.php(50): WP_Rocket\Plugin->load()
#2 /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-includes/class-wp-hook.php(307): rocket_init()
#3 /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-includes/class-wp-hook.php(331): WP_Hook->apply_filters()
#4 /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-includes/plugin.php(476): WP_Hook->do_action()
#5 /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-settings.php(461): do_action()
#6 /var/www/vasilis.rocketlabsqa.ovh/wp-config.php(123): require_once('...')
#7 /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-load.php(55): require_once('...')
#8 /var/www/vasilis.rocketlabsqa.ovh/htdocs/wp-blog-header.php(13): require_once('...')
#9 /var/www/vasilis.rocketlabsqa.ovh/htdocs/index.php(17): require('...')
#10 {main}
  • [x] 17. rocket_preload_revert_old_in_progress_rows is not deleted after deactivating/uninstalling the plugin: Note: commits are changing on the branch so we may not find all of the notes are on the latest commit or maybe the problem in another way like point 4

Mai-Saad avatar Jul 06 '22 11:07 Mai-Saad

@Mai-Saad @piotrbak the preload pass all raws to pending as expected for point 7.

CrochetFeve0251 avatar Jul 08 '22 11:07 CrochetFeve0251