Remove old documentation from scitools.org.uk

[jamesp]

Ok, where is this version of the iris docs (v1.9!) being published from?

https://scitools.org.uk/iris/docs/v1.9.0/html/index.html

Am I the only person who gets the wrong docs every time from google?

[tkknight]

I believe that is a static build of Iris v1.9 that is hosted via github pages. The project (https://github.com/SciTools/scitools.org.uk) is configured to use scitools.org.uk via the setting page: https://github.com/SciTools/scitools.org.uk/settings/pages (viewable only if you have the permissions)

All Iris documentation before Iris 3.0 is hosted there, Iris 3.0 and later are on https://scitools-iris.readthedocs.io/en/latest/

[jamesp]

Thanks @tkknight that makes sense now. SEO is a hard problem to solve, but should we consider making a plan to eventually deprecate these docs? From what I can deduce, rtd is better at SEO to ensure the latest docs appear first in google even though they host many versions.

Here are a few things that I think are confusing to new users. Google search for:

1. "iris regridding" -> 1st result is to iris 2.0 on scitools.org.uk. RTD 3.2 is 2nd in list.
2. "iris projection" -> 1st result is to iris 1.9 on scitools.org.uk
3. "iris plotting" -> 1st result is to iris 2.0 on scitools.org.uk. Second result is iris 0.9 on esc24.github.io. docs for 3xx don't even appear on the list.
4. "iris zonal mean" -> 1st result is to iris 1.9 on scitools.org.uk

[tkknight]

I would have thought google search would have eventually pointed to the latest version by now - but clearly something is stopping that.

There are no current plans to retire the old ones. It is not viable to migrate them to rtd's as they would need work to do this.

We could look to archive these docs as a bundle zip file for reference if needed (perhaps in an archive folder here https://github.com/SciTools/scitools.org.uk/tree/master/iris/docs). We could do this for all v2.4 and all previous versions.

This approach would break all links to older docs. We could add a link to the legacy docs from the latest docs for reference.

This may need a little more thought before any action is made incase there are other impacts.

[trexfeathers]

Team discussion: Iris 2.3 is still currently widely used at the UK Met Office, but will be moved onto 2.4 etc. in due course. When this happens, we think that will be the appropriate time to 'archive'/zip/... the older non-RTD docs.

[jamesp]

Removing the Peloton tag from this now, we had a good conversation about the possible actions.

[rcomer]

Yesterday I witnessed a colleague trying to find something in the v1.9 docs, so I'm coming around to @jamesp's way of thinking! Would it be much work to just get rid of everything before v.2.0?

[tkknight]

I've done some thinking on this. I have looked at how the directory structure is setup and noticed that Iris uses version_switcher.js to manage the pull down of versions to switch to (old docs, not on read the docs). This is a single file for all old Iris documentation, each version points to, meaning this file could be changed to do "something" that will help.

After a little experimentation with stylesheets and javascript I have a proof of concept that will add a warning to the top the page of all old Iris documentation by appending some custom html/css. Here is the single file change (branch, not a PR yet)

This change should look like this: (needs testing, hard to do without making the change!)

...and when the screen is a bit smaller it will word wrap:

Notes:

• The warning banner will not be visible if the user scrolls down on the page.
• The google search text that @jamesp noted via https://github.com/SciTools/iris/issues/4396#issuecomment-958949908, I think would include the banner and be visible as it links to pages, not a bookmark on the page.
• This does not archive the old documentation but does better inform the user it is old and makes a good stop gap until Iris 2.4 is not used in the Met Office.

Looks like the cartopy docs (linked via https://scitools.org.uk/) goes straight to their new read the docs page with no apparent link back to the older docs, although they are still accessible if you know the url, such as https://scitools.org.uk/cartopy/docs/v0.10/

Is this change a viable approach? We can still archive the old (non read the docs) at a later date.

[jamesp]

This seems like a good, practical, improvement to the current state. thanks @tkknight!

[jamesp]

scitools.org.uk now has a banner showing that the user is not looking at the most recent version: https://github.com/SciTools/scitools.org.uk/pull/247.

I have added this issue to v3.4.0 milestone. We should review at that time if it is appropriate to remove the v2 docs. cc. @wjbenfold, v3.4.0 release manager.

[tkknight]

Thank to @jamesp 🥇 for the speedy review and follow up web debugging. Now working as planned.

[edmundhenley-mo]

Hi all, just to flag that I still have the "need to manually ignore v1.9 from search engine results" as a pain-point - see example search below (v1.9 top Google hit for "iris cube data fill" - unfortunately same for "iris cube" :( ). I can't tell from above whether or not this is meant to have been fixed - please ignore if still known bug, to be addressed in future. Flagging mostly as this is a ~biggie pain-wise for me, and would be adding more +1s to the upvote if I could!

[edmundhenley-mo]

Corollary usability question to above. Asking here as ~related, and just to seek out "technically im/possible" info to my question below - happy to promote to proper issue if it's doable!

When arriving via search engine, as per above, say searching for "iris cube metadata", I can ~reliably arrive at the subpage I mean - e.g. the userguide/iris_cubes.html subpage, albeit typically not at the version I want, for reasons ^^^.

The banner link implemented in https://github.com/SciTools/scitools.org.uk/pull/247 helpfully flags the version issue (thanks!), but implementation means that no matter which subpage you're on, the URL in the banner merely redirects to the top-level stable version of docs, https://scitools-iris.readthedocs.io/en/stable/, rather than the current subpage.

Up until now I've often clicked on that banner link, then navigated back to desired sub-page. Bit painful.

Paying more attention to the RTD version switcher, thanks to this issue, I now see that the links there helpfully preserve the subpage suffix - see screenshot below, and link at lower left from hovering over v3.0.3.

Now I'm enlightened, I'll likely be using this RTD version switcher route much more, esp as gives finer-grained control on version - yay!

However for improving iris docs usability for former unenlightened me / other users in similar boat, is is technically / ~easily possible to have a similar convenience injection of current subpage@stable version into the stable warning banner URL?

This just a lazy nice-to-have, so NP if subpage structure has changed enough over pre-RTD / RTD versions that this isn't possible!

[trexfeathers]

Corollary usability question to above. Asking here as ~related, and just to seek out "technically im/possible" info to my question below - happy to promote to proper issue if it's doable!

See #5045.

[tkknight]

Thanks @tkknight that makes sense now. SEO is a hard problem to solve, but should we consider making a plan to eventually deprecate these docs? From what I can deduce, rtd is better at SEO to ensure the latest docs appear first in google even though they host many versions.

Here are a few things that I think are confusing to new users. Google search for:

1. "iris regridding" -> 1st result is to iris 2.0 on scitools.org.uk. RTD 3.2 is 2nd in list.
2. "iris projection" -> 1st result is to iris 1.9 on scitools.org.uk
3. "iris plotting" -> 1st result is to iris 2.0 on scitools.org.uk. Second result is iris 0.9 on esc24.github.io. docs for 3xx don't even appear on the list.
4. "iris zonal mean" -> 1st result is to iris 1.9 on scitools.org.uk

For all these searches Iris 3.x is now no lower than third on the list, which is good. ReadTheDocs is doing the Search Engine Optimisations (SEO) for us well.

[tkknight]

After some thought here is a way forward:

• [x] 1. Each Iris docs for all v2.* will be archived into a zip file with the corresponding version name and placed into a new archive folder under https://github.com/SciTools/scitools.org.uk/tree/master/iris/docs. For example: https://github.com/SciTools/scitools.org.uk/tree/master/iris/docs/archive/v1.0.zip. Done, see https://github.com/SciTools/scitools.org.uk/pull/256.

  • [x] https://github.com/SciTools/scitools.org.uk/pull/256

• [x] 2. Each version that will be archived will have its directory removed. This should (after some time) be reflected in search engine results such as google (ie that version is no longer on the results). For example, this folder and all contents would be removed https://github.com/SciTools/scitools.org.uk/tree/master/iris/docs/v1.0

  • [x] https://github.com/SciTools/scitools.org.uk/pull/257

• [x] 3. The latest Iris docs will need the index.html page updated to point to the new archive GitHub folder instead of https://scitools.org.uk/iris/docs/v2.4.0/. There may be other pages to update too (see quick search results). It may be preferred to have have a legacy docs page explaining a bit more context than just linking to the archive.

  • [x] https://github.com/SciTools/iris/pull/5064

• [ ] 4. Check back in a week or two to see if the search results only show the newer versions of iris

We need to test to ensure that each version zip is usable once downloaded, unzipped and accessed. A quick test shows this works ok (need to test more of them), all 26 versions total, ranging from 4.5MB to 16MB.

○ → ls -GGalhS *.zip | cut -c 19- 
  16M Nov 10 17:30 v1.9.0.zip
  14M Nov 10 17:30 v2.4.0.zip
  14M Nov 10 17:30 v2.3.0.zip
  14M Nov 10 17:30 v2.0.zip
  14M Nov 10 17:30 v1.8.1.zip
  13M Nov 10 17:30 v1.12.0.zip
  13M Nov 10 17:30 v1.11.0.zip
  13M Nov 10 17:30 v1.13.0.zip
  13M Nov 10 17:30 v1.10.0.zip
  13M Nov 10 17:30 v1.9.2.zip
  13M Nov 10 17:30 v1.9.1.zip
  12M Nov 10 17:30 v2.1.zip
  12M Nov 10 17:30 v2.2.1.zip
  12M Nov 10 17:30 v2.2.zip
 8.1M Nov 10 17:30 v1.8.0.zip
 7.5M Nov 10 17:30 v1.7.2.zip
 7.5M Nov 10 17:30 v1.7.3.zip
 7.5M Nov 10 17:30 v1.7.zip
 5.6M Nov 10 17:30 v1.6.zip
 5.6M Nov 10 17:30 v1.5.zip
 5.3M Nov 10 17:30 v1.4.zip
 5.3M Nov 10 17:30 v1.2.zip
 5.3M Nov 10 17:30 v1.3.zip
 5.2M Nov 10 17:30 v1.0.zip
 5.1M Nov 10 17:30 v1.1.zip
 4.5M Nov 10 17:30 v0.9.1.zip

Notes

• Hopefully GitHib will be ok with zip (binary) files as large as 16MB.
• We can add a disclaimer to the legacy download links to highlight it may not view perfectly.
• All steps will need to be done carefully via PR's of course.
• We could retain some versions if we chose such as v2.3 and v2.4 if we think users are still actively using it.

I will test the zips files as mentioned.

Please comment on this approach if you have any thoughts.

[edmundhenley-mo]

• Hopefully GitHib will be ok with zip (binary) files as large as 16MB.

@tkknight - from complaint messages from GitHub when I've inadvertently pushed up large binary files before, I believe the limit is 50 MB.

NB though I now tend instead to use Git LFS for this sort of use-case - storing large binaries. These are versioned in a separate repo, storing a pointer in the main repo, and automagically up/downloading versioned pointer target when required. So main repo doesn't get too big - but LFS'd files behave as if they're normal versioned files. Works quite nicely in most circumstances - one exception is within Docker, but suspect that's not required for old docs? Previews of LFS'd files don't always work as on a normal repo either - but AFAIK, as I think you say ^^^, you wouldn't get perfect viewing of a zipped file anyway if stored w/o LFS. Edit: Git LFS storage and bandwidth quotas here: https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-storage-and-bandwidth-usage

[trexfeathers]

We could retain some versions if we chose such as v2.3 and v2.4 if we think users are still actively using it.

The earliest version I am aware of being in active use is Iris v2.2.

[rcomer]

I don't think v2.3 ever made it into Met Office SSS.

[trexfeathers]

I think this is great. It seems unlikely that there would be demand for earlier versions, but this is a reasonably easy way to provide the docs just in case.

I'm available to review PR's as needed @tkknight 😊

[trexfeathers]

We did it 🎉