Regenerate existing images

[adamsilverstein]

High-level feature

Adds the ability to regenerate existing images in core via a user interface.

Overview

This is an overview of related issues. This may not necessarily be comprehensive, for a full list see all issues related to the "Regenerate Existing Images" module here. When opening a new issue related to this feature, make sure to add the [Module] Regenerate Existing Images label.

Phase 1

Phase 1.1

• [x] #489

Phase 1.2

• [x] #478
• [x] #479
• [x] #503

Phase 1.3

• [ ] #480
• [ ] #481

Phase 1.4

• [ ] #482
• [ ] #493
• [ ] #494
• [x] #495

Phase 1.5

• [ ] #490
• [ ] #483

Phase 1.6

• [ ] #484
• [ ] #485
• [ ] #491

Phase 1.7

• [ ] #486
• [ ] #487
• [ ] #488

Additional Details

Module proposal: #445

Purpose

The purpose of this module is to provide administrators with a user interface to regenerate images in core without the need for an additional plugin or use of WP-CLI. This makes the feature more accessible to non-technical users.

As the regeneration of images is time consuming, and cannot be performed in a single request or a few AJAX requests, a new background process infrastructure will be created to handle long running tasks.

Rationale

As part of an effort to improve performance within WordPress, the Performance team has introduced WebP images into core. Providing administrators with a means to regenerate their existing images to create WebP versions will improve performance across their site.

Additionally, when administrators install and activate a new theme, existing images may be served at incorrect sizes, which can have a negative impact on performance. By providing an accessible way to regenerate these images, we can ensure that the correct sizes are served on the frontend and improve the site's performance.

Scope

The module can be seen as 2 parts: 1) the background processing infrastructure, and 2) the admin user interface and job to regenerate images.

The background processing infrastructure will be created to handle long-running, time-intensive tasks. It will be decoupled from the logic to regenerate existing images and can be used for any types of jobs.

The Regenerate Images job will utilize the background processing infrastructure to handle the regeneration of images. This will work alongside with a new admin settings screen to trigger the regeneration of images.

Additionally, other entry points across WP Admin can be created to trigger the regeneration of one or more images more contextually: for example, a bulk action in the Media Library or Gutenberg controls to regenerate all images in a post.

As this is a large feature, it will need to be broken down into multiple phases that build from one another. Each phase of the feature will first be introduced into the Performance Lab plugin and released before the next phase for testing.

To help with the release a separate branch will be created to act as a feature trunk branch. This branch will be used to develop and review the feature in parts. Once the feature is complete the feature trunk branch will then be merged into the Performance Lab plugin.

The first phases will include creating the Background Processing Infrastructure, the Regenerate Images Job, and Admin settings screen. Each phase will first be released to the Performance Lab plugin for testing and ultimately later merged into WordPress core.

A breakdown of the phases can be seen below:

Phase 1: Implement backend infrastructure

• Implement the backend infrastructure to handle resource/time intensive tasks
• Implement the image regeneration process on top of the new infrastructure
• Create a dedicated admin page to initiate a global image regeneration

Phase 2a: Provide alternative image generation types

• Extend the admin page to offer controls to define a subset of image sizes to regenerate, such as images uploaded between 2 dates.
• Extend the WordPress media library to provide a bulk action where administrators can select specific images to regenerate
• Extend the WordPress admin post list screen to provide a bulk action where administrators can select specific posts where it’s images will be regenerated

Phase 2b: Gutenberg Controls

• Provide a Gutenberg Document Panel to allow administrators to regenerate all images for the edited post
• Provide a Gutenberg control for the featured image to regenerate the post thumbnail
• Provide a Gutenberg panel for blocks (such as the Image block) where administrators can resize the blocks currently displayed image

[mitogh]

Adding a document with a proposal for support of multiple mime types and a process to update existing images on the sites:

• https://docs.google.com/document/d/1Wrf4K1x7rxHEaQsQ6OLiI4R05fahMikoy3znkETv9Q0/edit?usp=sharing

Please take a look at the document and add comments with suggestions / questions or feedback.

[eclarke1]

@mitogh @adamsilverstein updating this issue to be a [Type] Overview as it will act as our main "epic" issue, and we will create sub-issues once we have finalised an approach. Does that work for you?

[mitogh]

@mitogh @adamsilverstein updating this issue to be a [Type] Overview as it will act as our main "epic" issue, and we will create sub-issues once we have finalised an approach. Does that work for you?

Thank you @eclarke1

[mitogh]

A current workaround for this is installing the plugin:

• https://wordpress.org/plugins/regenerate-thumbnails/

And running the process would trigger the hooks where the additional images would be created, this would also update the metadata of the images associated with each additional image.

[masteradhoc]

+1 for this one. i used @mitogh approach to regnerate them for already existing images - worked great!

[eclarke1]

@felixarntz @adamsilverstein @jjgrainger @bethanylang @eugene-manuilov as discussed, I have renamed this epic

[pbiron]

A current workaround for this is installing the plugin:

• https://wordpress.org/plugins/regenerate-thumbnails/

And running the process would trigger the hooks where the additional images would be created, this would also update the metadata of the images associated with each additional image.

Another "workaround" is to use WP-CLI, e.g.,

$ wp media regenerate

I've tested both methods and they work great.

[mitogh]

Thanks for taking a look and testing with WP CLI @pbiron

[mxbclang]

Noting that the document shared in this comment is outdated and we are working a new one that's still in progress. A new link will be shared here when ready for review.

[rjasdf]

What about the new <picture> tag? It allows multiple images to be listed - and the browser picks the optimal one for the screensize available.

[mitogh]

Good point <picture> has been considered and strategies to use it as replacement for the <img> tag is an option, however one of the major concerns is the fact that updating the markup out of the box for existing sites might break some styles in themes due is a different container and a different markup, the main idea would be to introduce a brand new function that produces picture elements to wrap the image in all the available formats, for new inserted images, for existing images the code would remain the same in order to avoid breaking changes during updates to this plugin or WordPress.

Thanks for your question @rjasdf

[rjasdf]

@mitogh - Sure; a new thingie makes sense. Do you happen to know what happens with <picture> when the end-user stretches/shrinks the browser/smartphone screen? That is, will it dynamically switch to a different sub-img inside the picture tag? Or does it simply stretch/shrink the image already picked, possibly leading a poor rendering?

[mitogh]

Good question @rjasdf in that scenario <picture> alone is not enough and you need to introduce srcset with each of your sources inside of picture so the appropriate image is rendered on a specific breakpoint or condition that you specify.

That might involve a wide range of combinations that might not be possible to define out of the box with WP (at least with the default sizes) but an API would be a starting point for theme developers to expand it to their custom needs.

[felixarntz]

@rjasdf @mitogh Just wanted to chime in here to refer to #21: Maybe your conversation would be more applicable to that issue than the one here?

[mitogh]

Totally I think that's a good place to follow the conversation @felixarntz

Thanks. cc @rjasdf

[lkraav]

Background Processing Infrastructure

Quite curious to see a huge wheel reinvention happening instead of making a bet on Action Scheduler https://github.com/woocommerce/action-scheduler

Issues search doesn't reveal any discussion https://github.com/WordPress/performance/issues?q=is%3Aissue+is%3Aopen+scheduler was it held in some other environment?

There's also https://github.com/deliciousbrains/wp-background-processing

What's the win rolling our own here?

[jjgrainger]

Hi @lkraav

Thanks for your comment.

This module was proposed as part of #445 and discussed during the July 26 performance chat, one of our weekly Slack chats. The plugins you've mentioned were also raised during the performance chat but the module was overall well received and approved to move forward with. If you would like to join in on any future discussions you can find links and times to the Slack channel at the Core Performance Make page.

We took a lot of inspiration from the WP Background Processing library which is also used in the Action Schedular plugin. We've made some changes to work within the Performance Lab plugin and eventually core, but overall follow the same approach used within these plugins.

[felixarntz]

@lkraav +1 to what @jjgrainger says. The proposed implementation here largely follows the ideas of the two projects you're referring to. We're not really reinventing the wheel, we're mostly aiming to slightly adjust that architecture in a way that it can be easily ported into WordPress core.