Cached metadata on new post creation

[youdidnt]

• [x] I've read and understood the contribution guidelines.
• [x] I've searched for any related issues and avoided creating a duplicate issue.

Please give us a description of what happened.

Meta data in new posts is wrong. Post url and og:image are incorrect.

Please describe what you expected to happen and why.

Problem is in state new_to_publish, if the post wasn’t saved earlier as a draft or wasn’t seen in preview mode then the meta data is incorrect. Yoast is using random images from post content as og:image and url to post is incorrect (example.com/?p=1234). After cache refresh (another post save) meta data is correct. Problem is only on cached version of page, I'm using WP Rocket 3.5.5.1.

How can we reproduce this behavior?

1. Go to create post page.
2. Add post content, title, featured image, choose category, add tags.
3. Publish post immediately.
4. You need to see cached version of page.

Technical info

• If relevant, which editor is affected (or editors):

• [ ] Classic Editor
• [ ] Gutenberg
• [x] Classic Editor plugin

• Which browser is affected (or browsers):

• [x] Chrome
• [x] Firefox
• [x] Safari
• [x] Other

Used versions

• WordPress version: 5.4.1
• Yoast SEO version: 14.1
• Gutenberg plugin version:
• Classic Editor plugin version: 1.5
• Relevant plugins in case of a bug: WP Rocket 3.5.5.1

• Tested with theme: Bimber

[Djennez]

@youdidnt so if I understand this correctly, WPRocket is showing a cached version of the draft of a page, after you've published it?

[youdidnt]

That's right, it looks like it. It looks like WP Rocket saving page cache before Yoast SEO updates the meta data. However, it seems that in sketch mode Yoast should get correct thumbnail for the post.

[Djennez]

@youdidnt can you please share your WP Rocket settings, as I am unable to reproduce the issue in any way.

[youdidnt]

{"cache_mobile":1,"purge_cron_interval":24,"purge_cron_unit":"HOUR_IN_SECONDS","remove_query_strings":1,"exclude_css":[],"critical_css":"","exclude_inline_js":[],"exclude_js":[],"defer_all_js_safe":1,"emoji":1,"sitemap_preload":1,"sitemaps":["https:\/\/example.com\/sitemap.xml"],"dns_prefetch":[],"cache_reject_uri":["\/test-cache\/(.*)"],"cache_reject_cookies":[],"cache_reject_ua":[],"cache_purge_pages":["\/news"],"cache_query_strings":[],"automatic_cleanup_frequency":"","cdn_cnames":["cdn.example.com"],"cdn_zone":["all","all"],"cdn_reject_files":["\/uploads\/(.*).pdf"],"heartbeat_admin_behavior":"","heartbeat_editor_behavior":"","heartbeat_site_behavior":"reduce_periodicity","google_analytics_cache":"1","cloudflare_api_key":"x","cloudflare_email":"[email protected]","cloudflare_zone_id":"x","cloudflare_protocol_rewrite":1,"sucury_waf_api_key":"","consumer_key":"x","consumer_email":"[email protected]","secret_key":"x","license":"x","secret_cache_key":"x","minify_css_key":"x","minify_js_key":"x","version":"3.5.5.1","cloudflare_old_settings":"","sitemap_preload_url_crawl":"500000","cache_ssl":1,"do_beta":0,"cache_logged_user":0,"do_caching_mobile_files":0,"minify_google_fonts":0,"minify_html":0,"minify_css":0,"minify_js":0,"minify_concatenate_css":0,"minify_concatenate_js":0,"defer_all_js":0,"embeds":0,"lazyload":0,"lazyload_iframes":0,"lazyload_youtube":0,"async_css":0,"database_revisions":0,"database_auto_drafts":0,"database_trashed_posts":0,"database_spam_comments":0,"database_trashed_comments":0,"database_expired_transients":0,"database_all_transients":0,"database_optimize_tables":0,"schedule_automatic_cleanup":0,"manual_preload":0,"do_cloudflare":0,"cloudflare_devmode":0,"cloudflare_auto_settings":0,"sucury_waf_cache_sync":0,"control_heartbeat":0,"cdn":0,"varnish_auto_purge":0}

WP Rocket support sait problem it may be that Yoast SEO is changing og:image after caching process.

[youdidnt]

I have "temporary" fix for that but problem is still there. And it is visible if you use auto posting on social media (for example Twitter) with the option Publish Immediately (I'm using SNAP plugin). However, on the post page, the meta data is correct. So invalid values are only for a few seconds after publication.

add_action('new_to_publish', 'new_publish_post_fix'); add_action('draft_to_publish', 'new_publish_post_fix'); function new_publish_post_fix() { if ( function_exists( 'rocket_clean_post' ) ) { rocket_clean_post( get_the_ID() ); } }

I hope this helps fix the error, but I still don't know if WP Rocket or Yoast SEO is the problem.

[youdidnt]

I have small update. Problem also occurs with other plugins like WP Super Cache. I have no problem with og:image but canonical url is wrong and displays the short wordpress link.

[Djennez]

Hmm, I've imported your settings and set up the same plugins as you've indicated in your original post (classic editor). I'm still not able to reproduce any of the problems you're describing.

[tmobilon]

Hi there,

I'm having the same problem here, with a similar configuration, but some different plugins.

Yoast is caching metadata from the Post Preview in object-cache. When the post is published, Jetpack sends it immediately to twitter, which then fetch the metadata from the url.

This process happens before Yoast gets the chance to purge the cached keys from the post preview metadata. And so, the URL is cached in NGINX with the wrong (post preview) metadata.

This bug is new, it started after 14.x update.

What i'm using:

• Yoast 14.3
• Wordpress 5.4.2
• Jetpack 8.6.1 with Publicize enabled (twitter posting)
• Nginx Helper Plugin 2.2.2
• Pressjitsu Redis Object Cache 1.1

How can we reproduce this behavior?

1. Enable some HTML cache;
2. Enable some Object cache;
3. Enable twitter auto post in "Publicize";
4. Write a draft, save and hit Preview;
5. Publish the post.

The page will go to the HTML cache with the wrong metadata. But if you add a random string to the URL, the metadata will be right.

One possible solution could be avoiding Yoast from storing the metadata from the post preview in the object-cache.

Other considerations:

1. The problem goes away when i disable Object Cache;
2. The problem persists with all Redis plugins.
3. With other Redis plugins it's even worse: the breadcrumbs are cached with a placeholder "DEFAULT";
4. This other guy is having the same problem, using W3 Total Cache + Yoast: https://wordpress.org/support/topic/preview-url-cached-in-canonical-meta-tag/

[tmobilon]

I can confirm that the problem is related to the new Yoast Inexables feature. I've installed Yoast Test Helper plugin, and Reseted/Deleted the indexables. And now the canonicals and thumbs (in twitter) are working again.

The downside is: our server load has increased almost 3x. Maybe because of the errors, since the table doesn't exists anymore:

PHP message: WordPress database error Table 'website_producao.wp_yoast_primary_term' doesn't exist for query SELECT * FROM wp_yoast_primary_termWHEREpost_id= '323280' ANDtaxonomy = 'category' LIMIT 1 made by require('wp-blog-header.php'), require_once('wp-includes/template-loader.php'), include('/plugins/amp/includes/templates/reader-template-loader.php'), AMP_Post_Template->load, AMP_Post_Template->load_parts, AMP_Post_Template->verify_and_include, include('/themes/tb/amp/single.php'), AMP_Post_Template->load_parts, AMP_Post_Template->verify_and_include, include('/plugins/amp/templates/html-start.php'), do_action('amp_post_template_head'), WP_Hook->do_action, WP_Hook->apply_filters, Yoast\WP\SEO\Integrations\Front_End_Integration->call_wpseo_head, do_action('wpseo_head'), WP_Hook->do_action, WP_Hook->apply_filters, Yoast\WP\SEO\Integrations\Front_End_Integration->present_head, Yoast\WP\SEO\Presenters\Abstract_Indexable_Tag_Presenter->present, Yoast\WP\SEO\Presenters\Twitter\Image_Presenter->g.."

If i may add, i believe this ticket shouldn't be a "severity:minor". Our organic traffic from Google News/Discover dropped ~90% because of the wrong canonicals. Now the traffic is back to normal at the same day, after this temporary fix.

A wrong canonical creates a infinite redirect chain:

1. Googlebot is pinged with the correct permalink and crawls the page.
2. The canonical points to a a non-friendly URL: /?p=post_ID
3. When googlebot fetches the non-friendly URL, WordPress redirects it to the correct permalink;
4. Back to step 1 and repeat.

We didn't get a single post featured in Google news box, since this bug started. And now it's back again.

So i can't run the indexables again, until there's a fix...

[tmobilon]

One more information: i thought that the information was being cached because of the twitter auto publish feature, but i was wrong. I've disabled the Publicize feature in Jetpack and reenabled indexables, and the issue with the canonicals is back.

So basically, it's a problem where the metadata from the post preview is going to the object cache, and not flushing in time for the first pageload. Then when the page is loaded, it holds the wrong canonical in the HTML cache. If you use a random string, the canonical is correct.

[herregroen]

Currently when building an indexable we always set the permalink. It seems that the interaction with various caching plugins may cause the wrong permalink to get cached.

To resolve this I believe we should not set the permalink when building posts that one of the following statuses: 'draft', 'pending', 'auto-draft', 'future'.

This will cause the permalink to get saved as NULL in which case it will automatically be calculated on page visit at which point it should be correct.

[tmobilon]

@herregroen Is it possible to apply the same behavior to all the metadata, as well? (Open Graph tags, schema markup, etc)

In some cases, the post thumbnail is returning the default setting. I even saw cases where the breadcrumb is compromised.

[DFC005]

Just adding my voice to this. I have the same problem as the OP using Yoast and WP Rocket.

We have resorted to updating all of our posts to make sure the og:image is set correctly. I have also seen instances where the og:url is not using the full canonical URL as it should as well.

[tmobilon]

@herregroen Hey there, any news on this issue?

I'm having to hit "update" on every post after we publish (almost 50 posts/day).

A fix to this issue would be much appreciated. :)

[tmobilon]

Update:

I've disabled the object cache, and it doesn't fix the issue. The canonical tag is still wrong, as well as the metadata. Posts are beeing published in twitter with the default image in the card.

So it's justa a conflict with any HTML caching plugin, since the indexables functions as metadata cache itself, insinde the database.

This bug is getting really annoying and there's absolutelly nothing that we can do to even mitigate it. Does Yoast has any intentions on fixing this soon?

[DFC005]

Yes, we're still constantly having to update every post before we can push it to social media to make sure it publishes correctly with the headline and feature image.

It's very annoying and timewasting.

[tmobilon]

Hi guys,

I really didn't wanna throw this card, but i'm starting to consider migrating to another SEO Plugin.

This bug seems like an easy fix, but still we're waiting for more than 3 months, and Yoast is not showing any interest in fixing it.

I got my entire editorial team hiting "update" everytime we publish a new post. And it's hard, because they need to do that in less than 3 minutes, or the post gets published on social media without any thumbs or with a broken URL (i don't know why but Facebook returns 404 to the "?p=" canonical, and shows no card).

People forget to hit update, then we need to check which posts were not updated and are still with the wrong metadata. I even asked my dev to develop some kind of routine to automatically update every new post.

We have more than 30 authors, publishing 50 posts per day. It's a complete mess and a huge waste of time!

Do you guys intend to fix this soon? If not, a simple response would be fine. At least then we can decide if we want to find another solution.

Thanks

[Djennez]

Since I am unable to reproduce the issue, it's a bit difficult to check. Is anyone here able to test if the Draft PR above fixes this issue for their site?

[tmobilon]

@Djennez Sure, i'll test it.

[tmobilon]

@Djennez With this fix, the canonical tag (and all the schema that relies on it) disappears completely. Then i need to hit update again, to make it show up.

This screeshot was made after hiting update. All of the parts in yellow were not there before updating.

[Djennez]

@mobilon thanks a bunch, that confirms my suspicions (unfortunately). Will have to investigate further.

[matpratta]

@Djennez I have tried your approach and played around with the file you mentioned (indexable-post-builder.php) and it seems Yoast is trying to create the indexable right when the "Add New" button is clicked for a post, which generates it with the wrong permalink (since no permalink still exists at that point).

Though the solution you suggested did, in fact, work when trying in our staging environment, it didn't work on production, which makes me believe this might be caused by other plugins being triggered at publish time, such as WP to Buffer Pro or Rank Math's Instant Indexing. If possible, I'd like to ask @youdidnt and @DFC005 to check their plugins to see if there's any sort of similar plugin that could be acting during publish time, since that might be the culprit.

[thespun]

Just wanted to add that this exact issue happened to us too. We wound up having to revert to 13.5 Premium to resolve it. Hoping it can be fixed in a future version.

[monbauza]

Please inform the customer of conversation # 653800 when this conversation has been closed.

[rmarcano]

Please inform the customer of conversation # 657071 when this conversation has been closed.

[tmobilon]

@Djennez Any news on this?

If the page is being generated before the metadata is updated, could it be a matter of changing the priority of the update hook?

Another possible fix: what if we skip the indexables for the first pageview, and deliver the metadata dynamically? Or maybe skip for the first X seconds after the post is published. This way, the old method would "step in" while the indexable is still updating with the correct metadata.

[mmikhan]

@Djennez I was able to reproduce the issue on my end using the Yoast SEO v15.3.

[mmikhan]

+1 for https://wordpress.org/support/topic/canonical-and-relnext-prev-problem/

[tmobilon]

I've made a test scheduling a post. We setup the schedule, and then we updated the post.

When it was published, the canonical was correct. Another scheduled posts had a wrong canonical.

@iamazik @Djennez Any news on this issue? Did you get the chance to read my previous comment, regarding the execution order of the function? https://github.com/Yoast/wordpress-seo/issues/15246#issuecomment-716941132

[Djennez]

@mobilon I've been trying different ways to reproduce this locally, but was not yet able to do so. However, I may have a workaround for you. Can you check if using the wpseo_dynamic_permalinks_enabled filter works around this issue for you? It'll just need to return true. This filter will disable the permalinks function of the indexables and may give you the proper permalinks.

Meanwhile, I have asked for other eyes-on on this issue.