rustdocs deployment design for substrate, polkadot, and cumulus

[jimmychu0807]

As a follow up on https://github.com/paritytech/ci_cd/issues/168

This issue is not specifically for this repository, but is to optimize rustdocs across substrate, polkadot, and cumulus for both internal and external dev.

Currently, we have a mess in the various documentations within paritytech.

This epic is to address the following issues:

• substrate docs are currently published to substrate.dev/rustdocs, crates.parity.io and now to paritytech.github.io/substrate

  • rustdoc that's published to substrate.dev/rustdocs is built in substrate-developer-hub/rustdocs instead of reusing the output from Substrate CI
  • no one knows what publishes to crates.parity.io
  • decide either to publish to these places from one process or to publish just to one place and have redirections to avoid potential unsynced docs

• define where to host the docs:

  • paritytech.github.io/substrate (in case of substrate it may be a redirection from
  • substrate.dev/rustdocs) paritytech.github.io/polkadot paritytech.github.io/ink etc with a table of contents in the root paritytech.github.io (there's a special repo for it paritytech/paritytech.github.io)
  • same, but on crates.parity.io/*

• setup federated search on the landing page

• post/fix the links to the documentation portal everywhere

@sacha-l @NukeManDan what thought do you have?

Moving https://github.com/paritytech/ci_cd/issues/176 to here cc @TriplEight

[TriplEight]

rustdocs deployment design for substrate, polkadot, and cumulus

I think we should target all our rust projects

[jimmychu0807]

@TriplEight Let's use this post as the design requirements for Parity-wide documentations / project web pages under paritytech.github.io.

• [ ] 1. Build an index page (just a list is good enough for first version) for rustdocs and projects at https://paritytech.github.io. Involved paritytech.github.io repo.

  • [ ] Check with @imadarai if we could use https://paritytech.github.io frontpage to showcase rustdocs and Parity projects.
  • [ ] Check any design/ux requirement on this cross-rustdocs frontpage. Company public-facing branding / design implication?

• [ ] 2. For each of the repo that we publish rustdocs, we can:

  Specify multiple branches and tags that we want to publish the rustdocs. For example, we can mark in .gitlab-ci.yml that when pushing/merging PRs on master, v3.0.0-monthly-2021-08, we build and publish the rustdocs pages and have them accessible at:

  • master: https://paritytech.github.io/substrate/master
  • v3.0.0-monthly-2021-08: https://paritytech.github.io/substrate/v3.0.0-monthly-2021-08

• [ ] 3. Have a latest symbolic link pointing to a specific branch/tag (a folder in gh-page). For example, we have latest pointing to v3.0.0-monthly-2021-08. Then accessing https://paritytech.github.io/substrate/latest will be the same as https://paritytech.github.io/substrate/v3.0.0-monthly-2021-08.

• [ ] 4. Have a simple index page at https://paritytech.github.io/substrate showing existing versioned rustdocs. Continuing the above example, the core of this html page will consist of the following list:

  • master
  • v3.0.0-monthly-2021-08 (latest)

  Visitors can click on one of the links to access the corresponding versioned rustdocs.

[TriplEight]

Build an index page...

Absolutely! Basic navigation is the best first step!

Check any design/ux requirement...

I think here we could ask @goldsteinsveta to chime in and shed some design light, but keep in mind that this is a fist step, so something simple would do.

For each of the repo that we publish rustdocs, we also have a mechanism to to mark multiple branches...

For now, I think it's OK to have it as is, that the page mentions the current version. But for later, I was thinking about implementing something like a header with a version choice as in https://docs.rs/libc/0.2.99/libc/

When docs.substrate.io domain is ready...

I do not know the details, do you want to have some redirection here?

[jimmychu0807]

@TriplEight I have spec'ed out the requirements I envisioned on Parity wide rustdocs at https://github.com/substrate-developer-hub/substrate-developer-hub.github.io/issues/1091#issuecomment-899326115

I Would like to build this out with you in Sep. I will particularly need your help on pt 2 and pt 3 on configuring the .gitlab-ci.yml. Also want to check how complex are those in implementation. I will help on getting pt 1 and pt 4 design and layout ironed out.

[TriplEight]

pt 2 is really easy, just a matter of mentioning a branch on which the job will run and passing its' name to the publishing host. pt 3 should be configured on the "portal" side. As for design/layout, I find https://docs.rs layout handy, here are some links.

[sacha-l]

@jimmychu0807: Happy to help with layout for and implementation for (1) and coordinate with the Design team. My question is, what's the difference between (1) and (4)? I think if we just add versioning to (1) like this https://docs.rs/ it solves for (4). Unless I'm misunderstanding what you have in mind.

The primary resources for customizing Rust crate landing pages are this: https://doc.rust-lang.org/nightly/rustdoc/the-doc-attribute.html and this: https://docs.rs/about. One idea would be to fork this: https://github.com/rust-lang/docs.rs which provides the basic infrastructure we're talking about.

[jimmychu0807]

@sacha-l

• pt1 happens in https://github.com/paritytech/paritytech.github.io repo
• pt4 is done in the respective project repos. Say for Substrate, it means there will be an index.html in the gh-pages branch in https://github.com/paritytech/substrate repo showing the available versioned rustdocs. Because we will have multiple version of rustdocs released for a project, say the master branch (for internal dev), the latest released tag, and a monthly tag, the eventual address of the rustdocs would be https://paritytech.github.io/substrate/<version>. If we don't have the index page, visitors will not know what version of rustdocs are available.

[sacha-l]

Makes sense. What next steps do you see for pt1 @jimmychu0807 ?

[imadarai]

@jimmychu0807 I checked with @ericmorel and this repo was just a staging area for several websites in past. I would perhaps only like to consult Security to make sure the approach is ok and we have their blessing. (using paritytech.github.io)

[jimmychu0807]

@imadarai I have message Kirill on this and getting his thought. Will further followup on this.

[TriplEight]

cc @jimmychu0807

Say for Substrate, it means there will be an index.html in the gh-pages

There is such a file almost everywhere where we publish, it can be customised.

[apopiak]

I'm not sure this is the correct place, but someone changed the url format for the rustdocs hosted on github.io. Can you please add a redirect from https://paritytech.github.io/substrate/<module path>.html to e.g. https://paritytech.github.io/substrate/master/<module path>.html so existing links keep working?

[TriplEight]

@apopiak this could've been an issue for substrate (with mentioning) or directly to ci_cd team. But anyway, take a look here: https://paritytech.github.io/substrate/. Isn't it what you're looking for?

[nuke-web3]

FYI migrated from https://github.com/substrate-developer-hub/substrate-developer-hub.github.io/issues/1091 to here

[imadarai]

@jimmychu0807 Just flagging this here in case it's not on your radar. Rust Docs - bottom two list items are going to 404. Let me know if I just happen to have a bad link.

[jimmychu0807]

This is still in my radar.

[nuke-web3]

related: #352

[TriplEight]

• [ ] not to forget have all the previous docs places to redirect to a single one: https://github.com/substrate-developer-hub/rustdocs should redirect to paritytech.github.io/substrate too, as well as docs.substrate.dev and crates.parity.io.

cc https://github.com/substrate-developer-hub/substrate-docs/issues/638