wordpress-develop icon indicating copy to clipboard operation
wordpress-develop copied to clipboard

Add output buffering for the rendered template

Open westonruter opened this issue 1 year ago • 2 comments

This PR introduces output buffering of the rendered template starting just before the template_redirect action. The output buffer callback then passes the buffered output into the wp_template_output_buffer filter for processing. This is reusing the same output buffering logic that was developed for Optimization Detective and Gutenberg's Full Page Client-Side Navigation Experiment.

Examples for how this can be used:

  • Always Load Block Styles on Demand: In classic themes a lot more CSS is added to a page than is needed because when the HEAD is rendered before the rest of the page, so it is not yet known what blocks will be used. This can be fixed with output buffering.
  • Always Print Script Modules in Head: In classic themes script modules are forced to print in the footer since the HEAD is rendered before the rest of the page, so it is not yet known what script modules will be enqueued. This can be fixed with output buffering.
  • Gutenberg's Full Page Client-Side Navigation Experiment: No longer would it need to start its own output buffer, but it could just reuse the wp_template_output_buffer filter.
  • Optimization Detective: The plugin would also be able to eliminate its output buffering, in favor of just reusing the wp_template_output_buffer filter.
  • Caching plugins would also not need to output buffer the response, but they could reuse the filter to capture the output for storing in a persistent object cache while also appending some status HTML comment.
  • Other optimization plugins (e.g. WP Rocket, AMP, etc) would similarly not need to do their own output buffering.

Trac ticket: https://core.trac.wordpress.org/ticket/43258


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

westonruter avatar Feb 26 '25 06:02 westonruter

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

Core Committers: Use this line as a base for the props when committing in SVN:

Props westonruter, flixos90, dmsnell, jorbin, peterwilsoncc.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

github-actions[bot] avatar Feb 26 '25 07:02 github-actions[bot]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

  • The Plugin and Theme Directories cannot be accessed within Playground.
  • All changes will be lost when closing a tab with a Playground instance.
  • All changes will be lost when refreshing the page.
  • A fresh instance is created each time the link below is clicked.
  • Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance, it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.

github-actions[bot] avatar Feb 26 '25 07:02 github-actions[bot]

@dmsnell:

While I may have left this comment before, I am very hesitant to support a built-in system which forces buffering of the entire output by default. This one decision essentially prevents any kind of streaming output from WordPress which otherwise might reduce latency to the client. […] So this is the big concern I have; not that code will choose to eliminate the ability to stream a response and get it out quicker, but because it ultimately prevents any plugin from streaming.

Thanks for the feedback and for raising this concern, which we did discuss a bit before. While the lack of streaming was indeed a potential drawback to output buffering in the past with classic themes, the reality is that now with block themes that ship has largely sailed. This is because all the blocks have to be rendered and then wp_head() runs. See template-canvas.php. This is great for performance because allows for scripts and styles to be enqueued which are actually used on the page, but it means the response cannot be streamed. So block-based theme templates are essentially using output buffering already, except without using ob_start(). So I do not see that adding document-level output buffering will introduce any significant latency in practice, not only due to how block themes work, but also due to page caching layers and/or other optimization plugins which are already doing output buffering (each in their own ad hoc way without any standardization).

westonruter avatar Sep 26 '25 21:09 westonruter

Thanks for the thoughtful response @westonruter and the link.

the reality is that now with block themes that ship has largely sailed. This is because all the blocks have to be rendered and then wp_head() runs. See template-canvas.php.

another way to look at this is that we introduced a regression there too, and I think we can look at that system for further optimization ideas. in a similar way that a browser starts with a full parser and a speculative parser, I bet WordPress could accomplish a lot of what it needs for enqueuing styles and scripts through a fast speculative parse, ship the HEAD, and then render the blocks.

this is something that the work in #9105 makes easier than ever, where we can quickly and efficiently process the block structure in a post before doing any real processing. that would, for instance, let us see every block type in use and check for things like block supports or even for the presence of CSS classes on a block’s “wrapping element.”


with the content type check I guess we are certain this won’t run on “REST” API calls? or RSS feeds, or XML-RPC calls?

dmsnell avatar Sep 28 '25 16:09 dmsnell

@dmsnell:

another way to look at this is that we introduced a regression there too, and I think we can look at that system for further optimization ideas. in a similar way that a browser starts with a full parser and a speculative parser, I bet WordPress could accomplish a lot of what it needs for enqueuing styles and scripts through a fast speculative parse, ship the HEAD, and then render the blocks.

I'd love it if we could do that, but I have doubts. For example, even if we have the full static block markup available for a fast analysis with a "preload scanner", we can't rely on that markup to anticipate what actually will be needed in terms of scripts and styles. This is because many blocks are dynamic and require PHP to render. Other blocks may be hidden entirely with render_block filters. Still others could be modified in arbitrary ways with filtering.

this is something that the work in #9105 makes easier than ever, where we can quickly and efficiently process the block structure in a post before doing any real processing. that would, for instance, let us see every block type in use and check for things like block supports or even for the presence of CSS classes on a block’s “wrapping element.”

For example, with Core-63676 we need to omit styles and scripts from being enqueued if a call to render_block() ends up not returning any markup (or an ancestor block is hidden). This all involves needing to render the entire page, including executing all PHP involved in rendering.

with the content type check I guess we are certain this won’t run on “REST” API calls? or RSS feeds, or XML-RPC calls?

That's right. The REST API runs before WordPress even loads the template-loader.php. When a REST API request is being made, it is served at the parse_request action, in which case the template_redirect action never fires. Similarly, with how the output buffer is started just before the template is included, it won't end up running for robots.txt, favicon, feeds, or trackbacks. Now, maybe this is not desirable and the output buffering should happen for in some of those scenarios. In particular, it may make sense for feeds to do some XML processing. It isn't needed for robots.txt requests since there is already a robots_txt filter. And favicon requests aren't relevant for output buffer filtering, since they just do redirects which can be overridden by the do_faviconico action.

XML-RPC requests wouldn't be included, since they use xmlrpc.php as the entrypoint, and not the regular WordPress execution flow via template-loader.php.

westonruter avatar Sep 29 '25 00:09 westonruter

See also Slack thread for additional discussion (and before in that core dev chat): https://wordpress.slack.com/archives/C02RQBWTW/p1759334728882769

westonruter avatar Oct 01 '25 16:10 westonruter

I made a test plugin to see to what extent streaming has today without output buffering:

<?php
/**
 * Plugin Name: Try output buffer and flushing
 */

add_action( 'wp_footer', function () {
	if ( isset( $_GET['flush_before_footer'] ) && rest_sanitize_boolean( $_GET['flush_before_footer'] ) ) {
		flush();
	}
	echo '<style>body { background: yellow; }</style>';
	if ( isset( $_GET['sleep_before_footer'] ) ) {
		sleep( (int) $_GET['sleep_before_footer'] );
	}
	echo '<style>body { background: lime; }</style>';
} );

When accessing the Sample Page which sleeps for 3 seconds before wp_footer without first flushing (sleep_before_footer=3&flush_before_footer=0):

https://github.com/user-attachments/assets/f5b2228c-8354-44fe-815e-e1ce279f4d22

When accessing the Sample Page which sleeps for 3 seconds before wp_footer after first flushing (sleep_before_footer=3&flush_before_footer=1):

https://github.com/user-attachments/assets/8462d86c-a880-4288-85f1-817f3e879f61

However, on another page with 100 paragraphs of Lorem Ispum, both with and without the explicit flush results in the same experience above the fold:

https://github.com/user-attachments/assets/f7837afe-c7a6-4ec6-9445-0b56b6522954

With Output Buffering Enabled

Here is the sample page, when the output buffer from this PR is enabled:

https://github.com/user-attachments/assets/5217aec6-7fc3-4584-9e84-1cc2db4e34dd

It's pretty close to the example of the Sample Page without the explicit flush. With the streamed version, the first auto-flushed chunk includes the site title.

However, when adding 100 paragraphs, then the experience is much different, and clearly worse since nothing is rendered until after wp_footer finishes:

https://github.com/user-attachments/assets/527a20e1-7d56-4c23-bd1f-71089684a1db

westonruter avatar Oct 01 '25 22:10 westonruter

The degraded streaming experience with enabling output buffering in these examples assumes that there is no plugin already doing output buffering, but this is already really common both for plugins to do. I did a search in WPDirectory for ob_start\(\s*[^)] and got the following (stale) relevant plugins that do output buffering:

Plugin Install Count
Elementor (image loading optimization module) 5,000,000
LightSpeed Cache 5,000,000
Wordfence Security 5,000,000
Really Simple SSL 5,000,000
EWWW Image Optimizer 1,000,000
CookieYes (script blocker module) 1,000,000
W3 Total Cache 1,000,000
WP Fastest Cache 1,000,000
Speed Optimizer 1,000,000
Autoptimize 1,000,000
WP-Optimize 1,000,000
Smush 1,000,000

Not included in this list is WP Rocket, which according to them has nearly 5 million websites.

The degraded experience with output buffering assumes there is no page caching in place. So even with the above caching plugins active which do output buffering today, streaming won't be relevant since a cached response will be served anyway (assuming there is a cached page available). The same goes for sites in which a reverse proxy is sending back cached responses, where there similarly won't be a degradation in the UX since the caching would preempt the output buffer.

westonruter avatar Oct 01 '25 22:10 westonruter

This is good engineering @westonruter — thanks for going through the effort to measure this stuff.

both with and without the explicit flush results in the same experience above the fold

This is expected when no user-space output buffering is applied, right? Once PHP’s internal buffer fills past a certain point it flushes automatically to the browser unless told not to via some user-space call to ob_start().

In my testing I found the default php -S 0.0.0.0 web server to flush once having sent 4,096 bytes to stdout.

The same goes for sites in which a reverse proxy is sending back cached responses, where there similarly won't be a degradation in the UX since the caching would preempt the output buffer.

This is mostly correct; at least with nginx this is disabled by calling header('X-Accel-Buffering: no'); to send the X-Accel-Buffering: no HTTP header.


Now lots of plugins are going to make the decision that it’s worth delaying the response in order to rewrite the top of the HTML document (the HEAD). That’s fine, normal, and reasonable.

In fact, it’s been my observation that most code is going to take an eager, high-latency, high-memory-overhead path as a default first reach.


a search in WPDirectory for ob_start(\s*[^)] and got the following

While I’m not sure entirely what this is supposed to demonstrate, it’s not that these plugins hold up render. Some of them will, but others, such as the EWWW Image Optimizer, appear to be using ob_start() locally within a single function to turn stdout output into a string to return, and in an admin page at that.

I bet we could detect this at large by estimating that if a response lacks a Content-length header it is being streamed whereas if it contains a content length it’s holding onto the full output before sending, or behind a cache. wordpress.org appears to stream its output.

But I still don’t know what that would tell us.

At some level I think we might be talking past each other. At least I am fairly sure I’m misunderstanding some things, so I will take a sit-out to see what others have to share. My questions and challenges are in good faith; I’m glad to see you working on this design.

dmsnell avatar Oct 02 '25 19:10 dmsnell

@dmsnell

This is expected when no user-space output buffering is applied, right? Once PHP’s internal buffer fills past a certain point it flushes automatically to the browser unless told not to via some user-space call to ob_start().

Yes, this is as expected.

While I’m not sure entirely what this is supposed to demonstrate, it’s not that these plugins hold up render.

I was attempting to look at the prevalence of plugins that buffer the output of the entire page. Granted, I may have done so naïvely.

Some of them will, but others, such as the EWWW Image Optimizer, appear to be using ob_start() locally within a single function to turn stdout output into a string to return, and in an admin page at that.

I'm not seeing this, at least in the version captured by WPDirectory:

// ...
	if ( $buffer_start ) {
		// Start an output buffer before any output starts.
		add_action( 'template_redirect', 'ewww_image_optimizer_buffer_start', 0 );
		if ( wp_doing_ajax() && apply_filters( 'eio_filter_admin_ajax_response', false ) ) {
			add_action( 'admin_init', 'ewww_image_optimizer_buffer_start', 0 );
		}
	}
}

/**
 * Starts an output buffer and registers the callback function to do WebP replacement.
 */
function ewww_image_optimizer_buffer_start() {
	ob_start( 'ewww_image_optimizer_filter_page_output' );
}

/**
 * Run the page through any registered EWWW IO filters.
 *
 * @param string $buffer The full HTML page generated since the output buffer was started.
 * @return string The altered buffer containing the full page with WebP images inserted.
 */
function ewww_image_optimizer_filter_page_output( $buffer ) {
	ewwwio_debug_message( '<b>' . __FUNCTION__ . '()</b>' );
	return apply_filters( 'ewww_image_optimizer_filter_page_output', $buffer );
}

I bet we could detect this at large by estimating that if a response lacks a Content-length header it is being streamed whereas if it contains a content length it’s holding onto the full output before sending, or behind a cache. wordpress.org appears to stream its output.

We could query HTTP Archive for how commonly WordPress pages are served with a Content-Length header to get a more definitive answer on that front for the ecosystem as a whole. But surely wordpress.org is behind some page cache, yeah? Surely it isn't streaming responses directly from the PHP application with every request. I see x-nc: HIT ord 1 which appears to indicate Fastly being used. Reverse proxies could be using dynamic assembly with ESI tags in which case a Content-Length would need to be omitted if it had been originally present in the response from WordPress. Then again, the lack of the Content-Length header could also indicate the response was originally streamed. Maybe we just can't know!

At some level I think we might be talking past each other. At least I am fairly sure I’m misunderstanding some things, so I will take a sit-out to see what others have to share. My questions and challenges are in good faith; I’m glad to see you working on this design.

I appreciate your thoughtful engagement on this issue, and that you're bringing to the fore important streaming considerations that I hadn't had top of mind.

Hopefully we can converge on a solution that addresses both of our important work-streams!

westonruter avatar Oct 02 '25 21:10 westonruter

@westonruter questions for your thoughts, having focused on this and explored the space more than anyone else might have:

  • do you think it’s possible to reframe this hook as a way to add progressive enhancements to the output? enhancements that would not leave the page broken if they didn’t run?
  • do you think it would be valuable to make this change?

I’m far less familiar with the performance plugin, but things like adding srcset to images seems like a simple enhancement whose absence only means potentially larger or potentially lower-quality images are displayed. Moving SCRIPT and STYLE elements around seems like another one of those things that if not performed, would still leave the page intact, just potentially slower to load.

apply_filters( 'wp_progressively_enhance_non_streamable_output_html', … )

Just musing here.

dmsnell avatar Oct 08 '25 19:10 dmsnell

@dmsnell Yes, the performance optimizations performed by Performance Lab features are enhancements on top of an otherwise non-broken page. So yes, the output buffer filter could be framed specifically as an optimization output buffer. That would indeed reframe the expectations that devs would have for what should or shouldn't be done with the filter. They should expect that it may not apply, so they shouldn't do anything critical for the content (e.g. hide stuff that should be behind a paywall).

My hesitation with this is for the non-optimization use case, namely for caching plugins to be able to have access to the page content for storage. Nevertheless, the reality is that they already hook into WordPress much earlier already in order to be able to serve back a cached response. For example, when WP_CACHE is true, WordPress core calls wp_cache_postload() in wp-settings.php after the plugins are loaded but before plugins_loaded fires:

https://github.com/WordPress/wordpress-develop/blob/2de7ed3bbc7263dd1f82ab43bd3a49cb46e6ae99/src/wp-settings.php#L569-L572

WP Super Cache defines wp_cache_postload() to start the output buffer right there (via wp_cache_phase() by default) or else it starts the buffer “late” at the init action (ref):

if ( isset( $wp_super_cache_late_init ) && true == $wp_super_cache_late_init ) {
	wp_cache_debug( 'Supercache Late Init: add wp_cache_serve_cache_file to init', 3 );
	add_action( 'init', 'wp_cache_late_loader', 9999 );
} else {
	wp_super_cache_init();
	wp_cache_phase2();
}

The wp-cache-config-sample.php includes:

$wp_super_cache_late_init = 0;

So every response in WordPress sites using WP Super Cache are going to be output buffered when caching is enabled.

In other words, caching plugins need to start the output buffers early in order to ensure the entire response is captured, after any potential nested output buffers are processed. This means that the inclusion of the wp_final_template_output_buffer action in this PR is simply not going to be useful for the intended purpose of offering up the output buffer to caching plugins. Since this PR starts the output buffer at a new action which runs after template_redirect action and after the template_include filter, any existing caching plugins would have opened their buffer already. The AMP plugin starts its output buffer at template_redirect:

/*
 * Start output buffering at very low priority for sake of plugins and themes that use template_redirect
 * instead of template_include.
 */
$priority = defined( 'PHP_INT_MIN' ) ? PHP_INT_MIN : ~PHP_INT_MAX; // phpcs:ignore PHPCompatibility.Constants.NewConstants.php_int_minFound
add_action( 'template_redirect', [ __CLASS__, 'start_output_buffering' ], $priority );

So if a caching plugin were to try using this wp_final_template_output_buffer action, the effect with the AMP plugin would be it would cache the page output before the AMP plugin performs its (expensive) optimizations, which entirely defeats the point. And it means it is highly unlikely that the ecosystem will ever shift all of their current output buffer processing to use this new action.

All this to say:

  1. I think we need to eliminate the wp_final_template_output_buffer action, since it's not useful for caching plugins as it was intended to be.
  2. The wp_template_output_buffer_html filter is renamed to something like wp_optimization_template_output_buffer_html, to make it clear it is for progressive enhancement.
  3. We introduce a new filter which allows the optimization output buffer to be turned off, but it remains on by default.

Also, surfacing https://github.com/WordPress/wordpress-develop/pull/8412#discussion_r2415117173 for the sake of Trac:

So how about this:

  1. The wp_template_output_buffer filter is eliminated.
  2. The HTML Content-Type detection is moved from wp_finalize_template_output_buffer() to wp_start_template_output_buffer().
  3. The wp_start_template_output_buffer() only starts the output buffer if the Content-Type is determined to be HTML.

In this way, the output buffer will only ever apply to HTML responses, leaving JSON, XML, and other content types free to stream.

westonruter avatar Oct 08 '25 22:10 westonruter

@dmsnell I've refactored this in a way that I think will suit both of our needs well. I implemented what I outlined above.

  1. The functions and filters are now explicitly mentioning that they are for optimization.
  2. The wp_template_optimization_output_buffer filter notes that any added filter callbacks must be for progressive enhancement for optimization, and that they must recognize that they may not apply.
  3. The output buffer is now only started by default if there are any wp_template_optimization_output_buffer filters added at the time of the template inclusion.
  4. The wp_template_output_buffered_for_optimization filter can force the output buffer to start for templates even when no filters have been added yet (for possible late-addition filters), or it can be used to force the output buffer off even when filters are present (e.g. for the sake of streaming applications).
  5. The output buffer now short-circuits if the response Content-Type is not HTML.

westonruter avatar Oct 10 '25 21:10 westonruter

@dmsnell:

thanks for all your persistence on this. it definitely reads more now like something which can open up the output to buffering but also speaks to the downsides to be aware of when doing that.

🎉

one note, which isn’t significant: I think we discussed the nuance of using this for optimization, but I don’t think it’s limited to that. if we wanted to continue playing with the names, I wonder if wp_finalize_template_enhancement_output_buffer would capture what you want while being less specific to page performance (for example, code adding other enhancements to the HTML, code performing analytics on the outbound HTML, etc…)

Great point. I've applied s/optimization/enhancement/ in 80d24ae2826ac9df7847a21827805027e8f143b6.

westonruter avatar Oct 11 '25 18:10 westonruter

I've updated the Always Load Block Styles on Demand plugin to use the latest state of this PR, including a new helper function wp_should_output_buffer_template_for_enhancement() and action wp_template_enhancement_output_buffer_started. I found these to be necessary when using the API to ensure that hooks aren't added when the output won't actually end up getting output buffered.

For the Twenty Twenty theme, this allows for the total amount of CSS on the page to be be go down from 252 kb to 148 kB, increasing usage from 13% to 20%:

Before:

Screenshot 2025-10-11 at 12 37 03

After:

Screenshot 2025-10-11 at 12 37 15

westonruter avatar Oct 11 '25 19:10 westonruter

I did some benchmarking on the performance impact for Always Load Block Styles on Demand with the changes in this PR using a classic theme (Twenty Twenty). I looked at the Sample Page, where the LCP element is textual.

Benchmarking Logic
  • I used the wordpress-develop environment.
  • My wp-config.php includes this line:
define( 'SCRIPT_DEBUG', isset( $_GET['script_debug'] ) ? (int) $_GET['script_debug'] : true );
  • I have an mu-plugin present for controlling which plugins are active via the query parameters: https://gist.github.com/westonruter/9c791f4f8cc1cc37e7b3f4bc2db9be97
  • I'm using GoogleChromeLabs/wpp-research to do the benchmarking with the following script:
number=100
before_url='http://localhost:8000/sample-page/?enable_plugins=none&script_debug=0'
after_url='http://localhost:8000/sample-page/?enable_plugins=always-load-block-styles-on-demand&script_debug=0'

npm run research -- benchmark-web-vitals --url="$before_url" --url="$after_url" --number=$number --network-conditions='broadband' --diff --output=md | tee broadband.md
npm run research -- benchmark-web-vitals --url="$before_url" --url="$after_url" --number=$number --network-conditions='Fast 4G' --diff --output=md | tee fast-4g.md
npm run research -- benchmark-web-vitals --url="$before_url" --url="$after_url" --number=$number --network-conditions='Slow 3G' --diff --output=md | tee slow-3g.md

For each emulated network condition, I did 100 requests without output buffering and then 100 requests with output buffering. As expected, the TTFB is degraded since the entire page has to be rendered prior to any bytes being served. Nevertheless, the LCP is significantly improved by ~20%. This is because even though TTFB is delayed, there are fewer render-blocking stylesheets to load once the HTML has been downloaded. Note in particular the LCP-TTFB metric for a broadband connection being ~30% improved, which is closer to what a site with page caching would experience.

Broadband:

Metric Before After Diff (ms) Diff (%)
FCP (median) 340.9 268.55 -72.35 -21.2%
LCP (median) 349.3 276.95 -72.35 -20.7%
TTFB (median) 20.8 45.4 +24.60 +118.3%
LCP-TTFB (median) 327.35 229.8 -97.55 -29.8%

Fast 4G:

Metric Before After Diff (ms) Diff (%)
FCP (median) 676.25 562.6 -113.65 -16.8%
LCP (median) 684.8 570.9 -113.90 -16.6%
TTFB (median) 20.6 44.1 +23.50 +114.1%
LCP-TTFB (median) 664.7 527.55 -137.15 -20.6%

Slow 3G:

Metric Before After Diff (ms) Diff (%)
FCP (median) 8999.55 7137.65 -1861.90 -20.7%
LCP (median) 8999.55 7137.65 -1861.90 -20.7%
TTFB (median) 22 46.95 +24.95 +113.4%
LCP-TTFB (median) 8977.7 7089.6 -1888.10 -21.0%

westonruter avatar Oct 11 '25 23:10 westonruter

Next I ran the benchmarking on all of the core classic themes, reducing the iteration count to 100 requests before/after and just emulating a broadband connection.

All themes show significant improvements to LCP.

Benchmarking Logic

The following is using https://github.com/GoogleChromeLabs/wpp-research/pull/199

#!/bin/bash

set -e

themes="
	twentyten
	twentyeleven
	twentytwelve
	twentythirteen
	twentyfourteen
	twentyfifteen
	twentysixteen
	twentyseventeen
	twentynineteen
	twentytwenty
	twentytwentyone
"
number=10
before_url='http://localhost:8000/sample-page/?enable_plugins=none&script_debug=0'
after_url='http://localhost:8000/sample-page/?enable_plugins=always-load-block-styles-on-demand&script_debug=0'

echo '' > 'all.md'

for theme in $themes; do
	echo $theme

	npm --prefix ~/repos/wordpress-develop run env:cli theme activate "$theme"

	echo "## $theme" >> 'all.md'
	npm --silent run research -- benchmark-web-vitals --url="$before_url" --url="$after_url" --number=$number --network-conditions='broadband' --diff --output=md |
		grep -v 'Success Rate' |
		sed '1s/|[^|]*|[^|]*|[^|]*/| Metric | Before | After /' |
		awk '
			BEGIN { FS=OFS="|" }
			NR==2 {
				for (i=3; i<=NF-1; i++) $i=" ---: "
			}
			NR!=2 {
				for (i=3; i<NF; i++) {
					gsub(/^[[:space:]]+|[[:space:]]+$/, "", $i);
					$i=" "$i" "
				}
			}
			1
		' |
		tee "$theme.md" |
		tee -a 'all.md'
	echo '' >> 'all.md'

	# TODO: It would be great if benchmark-web-vitals also included TTLB!

done

twentyten

Metric Before After Diff (ms) Diff (%)
FCP (median) 334.15 231.8 -102.35 -30.6%
LCP (median) 👉 334.15 258.3 -75.85 -22.7%
TTFB (median) 67.55 97.8 +30.25 +44.8%
TTLB (median) 95.1 133.6 +38.50 +40.5%
LCP-TTFB (median) 266.95 159.55 -107.40 -40.2%
TTLB-TTFB (median) 26.6 35.05 +8.45 +31.8%

twentyeleven

Metric Before After Diff (ms) Diff (%)
FCP (median) 377.7 293.1 -84.60 -22.4%
LCP (median) 👉 377.7 301.25 -76.45 -20.2%
TTFB (median) 69.55 94.3 +24.75 +35.6%
TTLB (median) 91.7 119.25 +27.55 +30.0%
LCP-TTFB (median) 308.4 206.15 -102.25 -33.2%
TTLB-TTFB (median) 21.7 24.25 +2.55 +11.8%

twentytwelve

Metric Before After Diff (ms) Diff (%)
FCP (median) 400.2 342.35 -57.85 -14.5%
LCP (median) 👉 499.85 441.05 -58.80 -11.8%
TTFB (median) 70.3 101.05 +30.75 +43.7%
TTLB (median) 97.8 139.95 +42.15 +43.1%
LCP-TTFB (median) 428.85 339.05 -89.80 -20.9%
TTLB-TTFB (median) 28.25 38.3 +10.05 +35.6%

twentythirteen

Metric Before After Diff (ms) Diff (%)
FCP (median) 477.15 405.5 -71.65 -15.0%
LCP (median) 👉 610.3 532.25 -78.05 -12.8%
TTFB (median) 59.85 84.5 +24.65 +41.2%
TTLB (median) 88.65 121.5 +32.85 +37.1%
LCP-TTFB (median) 548.6 447.1 -101.50 -18.5%
TTLB-TTFB (median) 28.7 36.5 +7.80 +27.2%

twentyfourteen

Metric Before After Diff (ms) Diff (%)
FCP (median) 458.7 383.6 -75.10 -16.4%
LCP (median) 👉 551.3 475.3 -76.00 -13.8%
TTFB (median) 58.9 83.65 +24.75 +42.0%
TTLB (median) 85.4 120.8 +35.40 +41.5%
LCP-TTFB (median) 492.4 392.3 -100.10 -20.3%
TTLB-TTFB (median) 26.8 37.1 +10.30 +38.4%

twentyfifteen

Metric Before After Diff (ms) Diff (%)
FCP (median) 491.65 418.4 -73.25 -14.9%
LCP (median) 👉 581.25 510 -71.25 -12.3%
TTFB (median) 58.1 84.1 +26.00 +44.8%
TTLB (median) 86.15 122.75 +36.60 +42.5%
LCP-TTFB (median) 524.4 424.7 -99.70 -19.0%
TTLB-TTFB (median) 27.9 38.1 +10.20 +36.6%

twentysixteen

Metric Before After Diff (ms) Diff (%)
FCP (median) 466.1 389.25 -76.85 -16.5%
LCP (median) 👉 554.85 475.1 -79.75 -14.4%
TTFB (median) 61.25 84.2 +22.95 +37.5%
TTLB (median) 90.05 123.25 +33.20 +36.9%
LCP-TTFB (median) 487.9 390.6 -97.30 -19.9%
TTLB-TTFB (median) 28.1 38.35 +10.25 +36.5%

twentyseventeen

Metric Before After Diff (ms) Diff (%)
FCP (median) 517.45 432 -85.45 -16.5%
LCP (median) 👉 524.2 456.5 -67.70 -12.9%
TTFB (median) 58.3 79.65 +21.35 +36.6%
TTLB (median) 136.7 176.9 +40.20 +29.4%
LCP-TTFB (median) 465.5 377.1 -88.40 -19.0%
TTLB-TTFB (median) 77.8 97.6 +19.80 +25.4%

twentynineteen

Metric Before After Diff (ms) Diff (%)
FCP (median) 472.6 394.8 -77.80 -16.5%
LCP (median) 👉 481 403.15 -77.85 -16.2%
TTFB (median) 57.8 84.1 +26.30 +45.5%
TTLB (median) 84.45 119.9 +35.45 +42.0%
LCP-TTFB (median) 422.8 318.6 -104.20 -24.6%
TTLB-TTFB (median) 26.05 35.4 +9.35 +35.9%

twentytwenty

Metric Before After Diff (ms) Diff (%)
FCP (median) 354.5 308.45 -46.05 -13.0%
LCP (median) 👉 362.9 317.2 -45.70 -12.6%
TTFB (median) 58.65 84.05 +25.40 +43.3%
TTLB (median) 92.65 130.35 +37.70 +40.7%
LCP-TTFB (median) 304.4 232.05 -72.35 -23.8%
TTLB-TTFB (median) 34.4 45.8 +11.40 +33.1%

twentytwentyone

Metric Before After Diff (ms) Diff (%)
FCP (median) 415.8 354.95 -60.85 -14.6%
LCP (median) 👉 415.8 354.95 -60.85 -14.6%
TTFB (median) 58.5 85.2 +26.70 +45.6%
TTLB (median) 90.65 126.3 +35.65 +39.3%
LCP-TTFB (median) 357.3 269.75 -87.55 -24.5%
TTLB-TTFB (median) 31 41.25 +10.25 +33.1%

westonruter avatar Oct 12 '25 00:10 westonruter

I had Gemini help me put together a script to benchmark classic theme performance in terms of Lighthouse performance scores with and without Always Load Block Styles on Demand, obtaining the median of 5 runs.

Average Relative Difference: +6.45
Average Percentage Difference: +7.74%

Theme Before Score (Median) After Score (Median) Relative Diff Percentage Diff
twentyten 97 100 +3 +3.0%
twentyeleven 95 99 +4 +4.2%
twentytwelve 87 95 +8 +9.1%
twentythirteen 77 85 +8 +10.3%
twentyfourteen 78 87 +9 +11.5%
twentyfifteen 77 85 +8 +10.3%
twentysixteen 80 88 +8 +10.0%
twentyseventeen 81 86 +5 +6.1%
twentynineteen 89 96 +7 +7.8%
twentytwenty 81 87 +6 +7.4%
twentytwentyone 91 96 +5 +5.4%

Lighthouse 12.8.2

westonruter avatar Oct 12 '25 05:10 westonruter

Testing batcache, I can see that it does cache unauthenticated REST API requests. I think it would be good to account for that to allow it to eventually migrate (I was testing with the Human Made variation).

If implementing for the REST API is overly complex for this PR, perhapes you could:

  • rename the functions & hooks to be generic (ie, remove the template references)
  • adding a context parameter to the various hooks with the response type: html, json, etc

@peterwilsoncc The description was out of date from the original purpose, which was to allow for this output buffer to be of use for page caches. Since then, the focus has sharpened to be specifically for enhancing HTML template responses. So it should not run for the REST API and it should not run for feeds or anything else that isn't HTML template responses that get loaded via the template_include filter. I've updated the description to be up-to-date.

westonruter avatar Oct 12 '25 23:10 westonruter

A commit was made that fixes the Trac ticket referenced in the description of this pull request.

SVN changeset: 60930 GitHub commit: https://github.com/WordPress/wordpress-develop/commit/9d03e8e15118cf2445fee04f567b09ee750cbaa4

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

github-actions[bot] avatar Oct 14 '25 00:10 github-actions[bot]

I don't understand why this PR was closed by committing r60930. Re-opening.

westonruter avatar Oct 14 '25 00:10 westonruter

@dmsnell Any further concerns?

westonruter avatar Oct 14 '25 22:10 westonruter

A commit was made that fixes the Trac ticket referenced in the description of this pull request.

SVN changeset: 60936 GitHub commit: https://github.com/WordPress/wordpress-develop/commit/a721cf9dc77ca24522fc9019130b0fa446ea69f9

This PR will be closed, but please confirm the accuracy of this and reopen if there is more work to be done.

github-actions[bot] avatar Oct 15 '25 17:10 github-actions[bot]

well done!

dmsnell avatar Oct 15 '25 23:10 dmsnell