Sitemap for Large sites with filter date/year

[advwebin]

Hello, The sitemap feature can be optimized a little more. It takes over 30 seconds to generate.

For large websites, like news websites which host a lot of old data, the sitemap generation can take a lot of processing power and waste resources.

I have seen this approach from metro.co.uk/sitemap.xml

They have developed https://github.com/Automattic/msm-sitemap in collaboration with automatic.

With this approach, you are hitting two issues.

1. Your crawl budget is optimized as Crawling takes place with only the newer or updated sitemaps.
2. Generation of new sitemaps is faster as it ads only the sitemap with a newer date, month, year. As per your setup.

[sybrew]

Thank you for the info and insights.

I see this as the only viable alternative to TSF's "optimized" sitemap, and it's well-suited for news sites.

Because the pagination is logically indexed by publication date, we can quickly find, clear, and ping old sitemaps individually on an archaic post's update.

Of course, this is for a niche sector, but I'm sure many people will appreciate it, even though they won't need it.