Map URL migration and plan redirects

[a-kyne]

Description

When permanently moving pages, create 301 "Permanently Moved" redirects for each URL. The redirects should be planned well in advance of the migration and mapped in a spreadsheet that includes the origin and the proposed destination URLs. The spreadsheet should document any URLs that will not be migrated, and should provide a reason for this decision.

When you move a URL, you must also be sure that you update all of the on-site, internal links that point to the old URL. If you do not update those links, search engines will continue to try to crawl the old URL and it may delay how quickly the new URL is indexed.

If large numbers of URLs will not be migrated, then the spreadsheet should document the expected loss of traffic over a 12-month period, and stakeholders should be asked to approve this information. This is an important step that should not be skipped because stakeholders who are pressed for time and resources will sometimes decide not to migrate chunks of website content, and then are unhappily surprised when they discover that the website has permanently lost all of the traffic that was being earned by the unmigrated pages.

When a page is not going to be replaced or updated in the new location, the URL should return a 404 "File Not Found" status code. Do not redirect it to the home page or to a category page because you want to retain the SEO value; Google will see that you have redirect a specific URL to a general one and reclassify the URL as a "soft 404".

If you wish to redirect an unmigrated URL to the home page or a category page because you believe that provides a better user experience than a 404 page would (debateable), then do so. It will not hurt your SEO; it just won't help it at all.

---

Success Criteria

• [ ] All old URLs were mapped one-to-one with new URLs or classified as 404
• [ ] All links to the old URLs were identified and plans to update are in place
• [ ] Lost traffic from deleted 404 URLs was projected and that number was approved by stakeholders

[stevejalim]

Porting over notes from https://github.com/mozmeao/springfield/issues/138

More to come, but tracking what we're discussing elswhere

• [x] Update refractr redirects - eg these
• [x] Add new redirects in Bedrock to send to Springfield for the pages we've moved
  • [ ] Watch for RTAMO and s=direct links for download page

LATER, not now:

• [x] Release notes -> wildcard/smarter/regexy redirect
  • [x] Watch for hard-coded URLs in the ex-Nucleus data
• [x] Review existing redirects.py in Bedrock and update to send to Springfield directly - if in the meantime we have some jumps within Bedrock, that's fine

[janbrasna]

And after all is stable, and the switches/settings removed, there are some final touches:

• If possible, stay on the same locale when going from wmo to fxc both in updated links, and the actual location response headers.
  • (now it goes either through root content negotiation and language choice, or on subpages picks locale from headers or a default en-US) — currently we plan to state that 301 for WMO /de/firefox/developer is FXC /(root)/channel/desktop/developer/ (no locale, that then 302s properly according to request) which it is not, the ultimate 301 for say crawler purposes would be the one with the explicit locale in the destination URL.
• Skip hitting bedrock from refractr for rules that can go to springfield directly.
  • Similarly to combing over old bedrock redirects to update some of the existing rules to point to fxc directly, there are rules in refractr that are sending traffic to WMO/firefox/new from many other domains so these can be also updated there to split WMO and FXC destinations and point accordingly directly. (i.e. mozilla.nl stays, getfirefox.de gets updated et al.) ~ I'll slowly start preparing these yaml changes.

[stevejalim]

Closing this - can reopen a smaller-scope ticket if we spot areas to improve