Page slugs and URLs: improve and clarify customization

[rlanzafame]

It is important to easily and reliably manage the URL's and slugs for website "pages." There is already a lot of functionality in the ecosystem, but the default behavior is not always desired. A common example is when moving from JB1 to JB2: authors want to ensure URLs for myst/JB2 book pages are identical to the "old" pages made with JB1. This has already been addressed in part with existing features, and there are also a number of Issues and feature requests. In this Issue I try to take a step back and propose a more holistic view for implementing and managing this, while also trying to summarize the prior issues and requests.

Proposal

Make it as easy as possible to customize URL's at book and page level by implementing a few simple options that can be defined inmystl.yml. Page level customization is preferably defined in the myst.yml file rather than the yml header to ensure that one can see everything in one place (i.e., hunting through each page to troubleshoot or understand the URL customization would be tedious).

To accomplish the example case described above, along with overriding the URL of a single file/page, the myst.yml would look like this:

...
site:
  template: book-theme
  options:
    urls_preserve_folders_and_files: true
...
- file: filename.ext
  override_url: make_my_file_have_this_url
...

Additional Information

Describing implementation in more detail.

Book-level URL customization

The option urls_preserve_folders_and_files is a single option would be equivalent to enabling at least the following two options, which could each be enabled separately, depending on the use case:

...
site:
  template: book-theme
  options:
    folders: true
    filenames: true
...

Side note: consider making the options easier to understand when the uninitiated read the yml file:

• folders --> urls_preserve_folders
• filenames --> urls_preserve_filenames

Note that this the folders option was implemented in #670 (described on this page of docs) and filenames is already described in Issue #1648. As indicated in #1648, there may be desired behavior that is complex to implement, or perhaps subjective, for example: an author wants to preserve filenames but automatically handle some parts of this like é becomes e instead of -. Ideally these options are specified as a sub-options to the main option, for example, if (in the future!) the functionality to do this were implemented in option convert_accents, it would be enabled optionally like this (please someone correct my yml syntax/structure, I am just guessing here!),:

...
site:
  template: book-theme
  options:
    folders: true
    filenames:
     enabled: true
     options:
      convert_accents: true
      other_cool_option: blah
      ...
...

Page-level URL customization

While the book-level customization with automatic adjustments to folders and filenames will be great, it would also be good to override this at a page level. This would be especially important, for example, if the URL's from only a few pages from the "old" book need to be preserved, while allowing the awesome automatic flattening/shortening features of MyST to remain in place. Or, being able to tweak the final URL if the existing features described above are not sufficient (e.g., change é to e manually before such an automatic option is implemented. Sometimes, there is unexpected behavior and an easy-to-apply backdoor fix would be very useful.

This should look like this in the toc:

- file: filename.ext
  override_slug: make_my_file_have_this_slug_but_folder_behavior_governed_by_other_options

As indicated, slug would only touch the filename change, not folder behavior. To completely modify URL after root/base:

- file: filename.ext
  override_url: make_my_file_have_this_url

This overrides any folder-level options specified at book level.

Warnings should be implemented to check whether the name overrides conflict with other URL's in the book and say so in the CLI output, as I believe this would not break build, and manifests itself it "missing pages" in the built book that are hard to figure out where they went.

Additional page-level customization in myst.yml

It seems to me that other things could be specified at the page level that should be listed with the toc, for example, the orthogonal features for redirecting, which are described in Issue #1472 (it is also in myst-theme repo Issue #413 and in jupyter-book/jupyter-book repo Issue #2328 and #1048, about rediraffe for JB1).

Implementing this in a way that allows for future customization (especially with user-defined plugins!) would be great, for example:

- file: filename.ext
  override_url: make_my_file_have_this_url
  cool_thing1: true
  disable_somthing: true
  variable_for_something_on_page_especially_with_thebe_or_jupyter_lite: 8

Facilitating customization at page level in the toc is something I affectionately refer to as "table of contents on steroids" :)

Thank you @parmentelat for pointing out places in the docs and older Issues to make it easier to track this down and summarize things more clearly!