Autoloading WordPress classes

[aristath]

trafficstars

Trac ticket: https://core.trac.wordpress.org/ticket/60414

---

This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

[tomdevisser]

Ohhh, love this PR!

[spacedmonkey]

Run some profiles, before and after, the results, are sadly not promising.

2023 theme

Before

After

2021 theme

Before

After

[aristath]

Run some profiles, before and after, the results, are sadly not promising.

Not quite... 😄 The numbers on the screenshots above point out something interesting: PHP time is increased by ~1.5%, but memory consumption is decreased by ~4.5%.

Worth noting: This is just the 1st step towards a much bigger project to modernize WP and make it more efficient, it's not the end of the road. There are many things we can do to improve and modernize WP, but none of that can happen if WP keeps loading everything, always. Implementing a simple autoloader doesn't have a big impact in itself (the benefits and costs are marginal, and one could argue that a 4.5% decrease in memory-cost is more important than a 1% increase in time-cost), but it does open the door to many other improvements 👍 The real performance improvements will come after the autoloader has landed, where we'll be able to take advantage of it. This PR just adds the necessary infrastructure for the future 😉

[lkraav]

Implementing a simple autoloader doesn't have a big impact in itself (the benefits and costs are marginal, and one could argue that a 4.5% decrease in memory-cost is more important than a 1% increase in time-cost), but it does open the door to many other improvements +1

I'm also thinking that at very least, it's a DX improvement and future unlocker. If it lands with no performance regressions, that's cherry on top.

[lkraav]

Hey @aristath are you daily driving this by any chance, or is anybody else?

Wondering if I could become real hands-on for testing this in my semi-production / staging env :thinking: or will everything blow up instantly.

[aristath]

Hey @lkraav!

The implementation here is complete, and the autoloader works as expected. I haven't seen anything "blow up"... Once/week I rebase the PR and update it, pulling the latest changes from Core, updating anything that needs updating, resolving conflicts etc, keeping it up to date with WP trunk. This definitely needs lots of testing, so if you use it on your semi-production/staging sites that would be great, and it would allow us to further polish the implementation! 🙇

I plan to create a new trac ticket when the time comes and publish a formal proposal, so any testing and additional data we have will be greatly appreciated! 👍

[lkraav]

Once/week I rebase the PR and update it, pulling the latest changes from Core, updating anything that needs updating, resolving conflicts etc, keeping it up to date with WP trunk.

Ah, this brings up a compatibility question - I'd like to stay in at least release candidate stability, but I guess this branch is constantly moving with trunk? Maybe you have tags available so I could merge only up to some "latest compatible with 6.1, or 6.2" state?

[aristath]

I'd like to stay in at least release candidate stability, but I guess this branch is constantly moving with trunk?

Yeah, I'm keeping it up to date with trunk constantly... However, the patch should apply cleanly to RC!

[lkraav]

I'd like to stay in at least release candidate stability, but I guess this branch is constantly moving with trunk?

Yeah, I'm keeping it up to date with trunk constantly... However, the patch should apply cleanly to RC!

I'm running into trouble trying to apply it to release WP at https://github.com/WordPress/WordPress.git didn't think of that before.

src/ would be easy to strip, but looks like I also need to skip tests :thinking:

EDIT git apply 3470.patch --exclude=tests/* --exclude=phpunit/* -p2 --reject :muscle:

This hunk gets rejected on top of 6.2 RC2

$ [-] cat index.php.rej 
diff a/index.php b/index.php    (rejected hunks)
@@ -19,9 +19,8 @@
 
 /*
  * Load the actual index.php file if the assets were already built.
- * Note: WPINC is not defined yet, it is defined later in wp-settings.php.
  */
-if ( file_exists( ABSPATH . 'wp-includes/js/dist/edit-post.js' ) ) {
+if ( file_exists( ABSPATH . WPINC . '/js/dist/edit-post.js' ) ) {
        require_once ABSPATH . '_index.php';
        return;
 }

Aha but I see this index.php is build-only, so perhaps all good!

Skimmed through our sites, and everything seems to be working :muscle: no crashes!

[kasparsd]

@spacedmonkey The front-end performance test suite for this pull request appears to show a performance improvement for both classic and block themes:

That's with 20 runs against the trunk baseline.

[spacedmonkey]

This file will need to be broken up into multiple files. https://github.com/WordPress/WordPress/blob/10838cb5a009a95de13fa4d2f50afabcb7ca26c7/wp-includes/class-wp-block-parser.php

[spacedmonkey]

@kasparsd Not sure how you are getting those numbers.

I just run it locally again, 300 times, using the core performance teams tool. Running 300 times.

Trunk Response Time (median),389.06

This PR Response Time (median),398.22

That is 10ms slower. I am keeping on eye on this PR, I am sure can fix this performance issue.

[spacedmonkey]

@aristath I think I worked out what the performance regression is!

Before, simplePie was conditionally loaded. See.

Now it is always loaded. See

Simplepie is only loaded in fetch_feed. Maybe we should just load it there.

[aristath]

@spacedmonkey thank you for the additional notes! 👍

This file will need to be broken up into multiple files. https://github.com/WordPress/WordPress/blob/10838cb5a009a95de13fa4d2f50afabcb7ca26c7/wp-includes/class-wp-block-parser.php

I already have a track ticket for that (with a patch) in https://core.trac.wordpress.org/ticket/57832, and in Gutenberg https://github.com/WordPress/gutenberg/pull/48693

@aristath I think I worked out what the performance regression is!

Before, simplePie was conditionally loaded. See.

Now it is always loaded. See

You're absolutely right! Should be fixed now 👍

[spacedmonkey]

@aristath That change was big a deal. See profiles

Trunk

This PR.

[aristath]

@aristath That change was big a deal. See profiles

Nice!! So basically the autoloader now improves loading time, CPU time, and memory consumption all in one 🎉

[spacedmonkey]

Did some basic benchmarking of this PR. See this google doc.

In short, this improves response time, but improves are very small.

[kraftner]

Does this already have an associated Trac ticket or is it still so young that it kind of "flies under the radar"?

[aristath]

Does this already have an associated Trac ticket or is it still so young that it kind of "flies under the radar"?

No trac ticket associated with this PR yet... Soon though 😉

[spacedmonkey]

FYI https://github.com/WordPress/wordpress-develop/commit/ac87b8701f5c97afe59ac559479436d9a03035ac

[github-actions[bot]]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• The Plugin and Theme Directories cannot be accessed within Playground.
• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance, it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

[github-actions[bot]]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props aristath, sergeybiryukov, schlessera, spacedmonkey, westonruter, tjnowell, needle, tobiasbg, cybr, peterwilsoncc, costdev, soean, tfrommen, kaloyan, dd32, thomasdevisser, lkraav, kasparsd, kraftner, bartj.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[aristath]

Adding some numbers from tests ran today.

Included files

Adding this snippet in mu-plugins/snippets.php:

<?php
add_action( 'shutdown', function() {
	var_dump( count( get_included_files() ) );
} );

I got the following numbers:

	Trunk	Autoload	Diff
Front	442	347	-95 files (-21.5%)
Dashboard	479	370	-109 files (-22.7%)

I only checked the frontpage and the main dashboard page, so these numbers will differ depending on the context. Some pages will have smaller differences, others will have larger. I expect that REST-API endpoints will have a significantly larger benefit, but I don't know exactly how to measure that.

Memory usage.

Tested using the Debug Bar community plugin. Numbers in Kb

	Trunk	Autoload	Diff
Front	4327	4329	0.04%
Dashboard	4074	4078	0.09%

This shows that the autoloader implementation currently uses 0.04-0.09% more memory. I don't know if that is because I have XDebug running (which admittedly is a performance hog), these numbers contradict previous tests both by @spacedmonkey in https://github.com/WordPress/wordpress-develop/pull/3470#issuecomment-1490083546 as well as previous tests of mine. Worth examining what changed these past few months.

Time

Tested using XDebug profiling. Report built using webgrind. These numbers are the average of 5 runs because XDebug-profiling produces rather inconsistent numbers.

	Trunk	Autoload	Diff
Front	392ms	377ms	-3.82%
Dashboard	317ms	295ms	-6.94%

---

The above tests are somewhat basic. If someone else has the ability to run some more detailed tests to get more consistent numbers, it would be greatly appreciated 👍

[bart-jaskulski]

I have played a little with benchmarking, doing 100 sequential requests to each endpoint. I've expected that results would be more consistent, but I'm not sure what is the reason for a quite large variance (look at the standard deviation value for timing).

Benchmarking was running against PHP 8.1 with nginx server. Profiling data comes from xhprof (slightly better than xDebug and provides actual memory usage info).

Average Run Times (mean) / (median) / (stddev)

	trunk	autoload	Percentage Diff
admin	125.79ms / 121.37ms / ±22.38	125.02ms / 121.66ms / ±17.82	-0.62%
front	228.13ms / 223.95ms / ±13.45	230.23ms / 224.68ms / ±16.14	0.92%
rest	91.55ms / 89.32ms / ±13.23	87.20ms / 86.25ms / ±5.17	-4.75%

Average Memory Usage (mean) / (median) / (stddev)

	trunk	autoload	Percentage Diff
admin	6.04M / 6.04M / ±0.04	6.23M / 6.23M / ±0.00	3.08%
front	6.66M / 6.66M / ±0.00	6.69M / 6.69M / ±0.00	0.49%
rest	5.68M / 5.68M / ±0.00	5.71M / 5.71M / ±0.00	0.5%

I've left notes on how my benchmark was conducted, along with collected results in repository: https://github.com/bart-jaskulski/wordpress-benchmark

[aristath]

Since I lack the ability to test memory consumption reliably, I think a good measure would be the test-suite results posted in this PR.

https://github.com/WordPress/wordpress-develop/actions/runs/8264293168

• There seems to be virtually no difference in the memory before/after this PR.
• The differences in load-times is ±1ms in almost all areas when ran in the test-suite.

[kktsvetkov]

Have you considered having multiple smaller autoload.php files instead of just one big src/wp-includes/class-wp-autoload.php ? This way each module will have its own autoload map. Those issues with simple-pie and parser.php having multiple classes in them will be addressed this way, with a very local approach. In theory, having smaller autoloads will also help to test each module in isolation without relying on the global core autoload.

[kktsvetkov]

I'm curious if you have explored bundling Composer\Autoload\ClassLoader instead of building a new autoloader? It's more or less the standard nowadays, it's battle-tested, and it already supports working with classmaps

$composer = new \Composer\Autoload\ClassLoader();

 $composer->addClassMap([
	'wp_hook'    => 'wp-includes/class-wp-hook.php',
	'wp_locale'  => 'wp-includes/class-wp-locale.php',
]);

$composer->register();

It seems to me there will be an additional benefit as there are a lot of plugins and themes that are already using composer, although in a local capacity.

[tomjn]

I'm curious if you have explored bundling Composer\Autoload\ClassLoader instead of building a new autoloader? It's more or less the standard nowadays, it's battle-tested, and it already supports working with classmaps

@kktsvetkov acutely aware, discussion of it completely derailed the last attempt to include composer setting things back years. Having said that there was discussion of using composer to generate the file mappings, but you're best referring to the Make WP post for specifics, this isn't the place for it

[kktsvetkov]

I'm not suggesting using composer as it is used in general, instead only utilizing the ClassLoader as available solution. I did manage to read the previous thread, and you are right - the composer discussion did seem to get out of hand.

[aristath]

Added an additional check in site-health.

If there are any overriden classes, it will show a critical security warning.

To test this, I added a file in mu-plugins/w.php with the following contents:

<?php
class WP_Community_Events {}
class WP_Users_List_Table extends WP_List_Table {}

We can fine-tune the text, presentation etc, but I believe this can serve us well and inform users if any Core classes are being overriden by a plugin/theme/host.