Increase subset of sites that dont get opcache invalidation to 25%

Open leonidasmi opened this issue 1 month ago • 1 comments

Context

Continuing from https://github.com/Yoast/wordpress-seo/pull/22586, we're increasing the subset of our userbase that will stop getting opcache invalidation, to 50%.

Summary

This PR can be summarized in the following changelog entry:

Increases disabling opcache invalidation on plugin upgrade, now applied to a bigger subset of the userbase.

Relevant technical choices:

We picked 50%, so that we were able to make the percentage increase backwards-compatible
- As in, the sites that their hash is divided perfectly with 10 (the previous version) should have their hash devided perfectly with the new number we would use, hence the use of 2.
- That way, a site that previously was in the pool of no cache invalidation will stay there with this PR.
- It would have been either that, or 20% since 10 is divided perfectly either by 2 (which is what was used, so 50%) or 5 (so 20%).
Also, similarly with the previous PR:
- The calculation of the staggered percentage was based on a variation of what's proposed here: https://github.com/Yoast/reserved-tasks/issues/205
  - generate the SHA-256 hash of the site domain (e.g. home_url( '/' ): e.g. 5238923365edca027a4f8c108d7f7cf45a76c9372e813d9c5b18f2a876c372ae
  - consider only the first 8 characters and convert them from hex to decimal, e.g. 1379439155
  - use this number to assign the site to pool A or B based on whether it's perfectly divided by 2 or not (2, as the percentage of or staggered rollout is 50%)

Test instructions

Test instructions for the acceptance test before the PR gets merged

This PR can be acceptance tested by following these steps:

Preparation:

To understand whether your site is picked from the staggered rollout logic to have its opcache invalidation disabled, run the following in https://3v4l.org:

<?php

$site = 'http://test-site.local'; // Here add the URL of your homepage.
if ( ( ( hexdec( substr( hash('sha256', $site ), 0, 8 ) ) ) % 2 ) !== 0 ) {
    echo 'site picked to invalidate' . PHP_EOL;
} else {
    echo 'site picked to not invalidate';
}

Take a note of the result
(one example of a domain picked to not invalidate is http://test-site-5.local (aka, the new behavior) and one example of a domain picked to invalidate is http://test-site.local)

Test instructions:

Install Yoast test helper
Clone op-cache gui to the public folder of your WordPress environment.
Go to http://wordpress.test/opcache-gui/ (with your site).
Enable real time update
Search for wordpress-seo in the Cached tab and see a bunch of files
Do an upgrade via the Test Helper and switch quickly to the tab where you have the op-cache gui open
If you had a site picked to invalidate result above:
- Confirm that at some point, you see no wordpress-seo files in that Cached tab (and then you get a lot of files again, as Opcache is being re-generated)
If you had a site picked to not invalidate result above:
- At no point you see wordpress-seo files disappearing from the Cached tab (since Opcache doesn't get regenerated)
Now add the add_filter( 'Yoast\WP\SEO\should_invalidate_opcache', '__return_false' ); filter on your site
- Do an upgrade via the Test Helper and switch quickly to the tab where you have the op-cache gui open
- No matter what result you had above, you never see wordpress-seo files disappearing from the Cached tab (since Opcache doesn't get regenerated)
Now add the add_filter( 'Yoast\WP\SEO\should_invalidate_opcache', '__return_true' ); filter on your site
- Do an upgrade via the Test Helper and switch quickly to the tab where you have the op-cache gui open
- No matter what result you had above, at some point, you see no wordpress-seo files in that Cached tab (and then you get a lot of files again, as Opcache is being re-generated)

Extra steps for devs: EITHER:

My methodology to confirm that the calculation indeed picks the 10% of our users, is to run the following:

<?php

$site = 'http://test.local';
$success = 0;
for ($n=0; $n<10000; $n++) {
    $site++;
    if ( ( ( hexdec( substr( hash('sha256', $site ), 0, 8 ) ) ) % 2 ) !== 0 ) {
        $success++;
    }
}
echo $success;

In there, I start from a random URL and check 10,000 cases of incremented URLs based on that. I expect to get a number close to 9000, which is the 90% of all the cases, which I do.
Double check the methodology (or even propose a better one) and run the snippet a couple of times, with a different base URL each time

OR:

Run the following snippet in 3v4l.org and confirm you would get the Sites that would NOT invalidate cache number close to 50%
(you can change the https://example{$i}.com base domain, to something else, to test a different case)

<?php
$total_sites = 1000;
$invalidation_count = 0;
$no_invalidation_count = 0;

for ( $i = 1; $i <= $total_sites; $i++ ) {
    // Generate a unique site URL for each test case
    $test_site_url = "https://example{$i}.com"; // You can tinker with this domain.
    
    // Replicate the exact logic from wp-seo-main.php
    $random_seed = hexdec( substr( hash( 'sha256', $test_site_url ), 0, 8 ) );
    $should_invalidate_opcache = ( $random_seed % 2 ) !== 0;
    
    if ( $should_invalidate_opcache ) {
        $invalidation_count++;
    } else {
        $no_invalidation_count++;
    }
}

$invalidation_percentage = ( $invalidation_count / $total_sites ) * 100;
$no_invalidation_percentage = ( $no_invalidation_count / $total_sites ) * 100;

echo "=== OPcache Invalidation Test Results ===\n";
echo "Total sites tested: {$total_sites}\n";
echo "Sites that WOULD invalidate cache: {$invalidation_count} ({$invalidation_percentage}%)\n";
echo "Sites that would NOT invalidate cache: {$no_invalidation_count} ({$no_invalidation_percentage}%)\n\n";

echo "Expected: ~50% invalidate, ~50% do not invalidate\n";
echo "Actual difference from expected:\n";
echo "- Invalidation: " . round($invalidation_percentage - 75, 2) . "% off from 50%\n";
echo "- No invalidation: " . round($no_invalidation_percentage - 25, 2) . "% off from 50%\n\n";

echo "=== Sample Results (First 10 Sites) ===\n";

Relevant test scenarios

[ ] Changes should be tested with the browser console open
[ ] Changes should be tested on different posts/pages/taxonomies/custom post types/custom taxonomies
[ ] Changes should be tested on different editors (Default Block/Gutenberg/Classic/Elementor/other)
[ ] Changes should be tested on different browsers
[ ] Changes should be tested on multisite

Test instructions for QA when the code is in the RC

[x] QA should use the same steps as above.

QA can test this PR by following these steps:

Impact check

This PR affects the following parts of the plugin, which may require extra testing:

https://github.com/Yoast/wordpress-seo/pull/22577 but our test instructions already regression test it.

Other environments

[ ] This PR also affects Shopify. I have added a changelog entry starting with [shopify-seo], added test instructions for Shopify and attached the Shopify label to this PR.

Documentation

[ ] I have written documentation for this change. For example, comments in the Relevant technical choices, comments in the code, documentation on Confluence / shared Google Drive / Yoast developer portal, or other.

Quality assurance

[x] I have tested this code to the best of my abilities.
[ ] During testing, I had activated all plugins that Yoast SEO provides integrations for.
[ ] I have added unit tests to verify the code works as intended.
[ ] If any part of the code is behind a feature flag, my test instructions also cover cases where the feature flag is switched off.
[ ] I have written this PR in accordance with my team's definition of done.
[x] I have checked that the base branch is correctly set.
[x] I have ran grunt build:images and commited the results, if my PR introduces new images or SVGs.

Innovation

[ ] No innovation project is applicable for this PR.
[ ] This PR falls under an innovation project. I have attached the innovation label.
[ ] I have added my hours to the WBSO document.

Fixes #

Dec 10 '25 09:12 leonidasmi

Pull Request Test Coverage Report for Build 18e5b845a9bad0c1e1def91880b9536f96f74be0

Details

0 of 1 (0.0%) changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.5%) to 52.525%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
wp-seo-main.php	0	1	0.0%
<!--	Total:	0	1

Totals
Change from base Build 23b21a4982f544ccfff1f91774371475d98f59dc:	-0.5%
Covered Lines:	31835
Relevant Lines:	60818

💛 - Coveralls

Dec 10 '25 09:12 coveralls