Support uploading to GitHub pages using assets & artifacts instead of branches

[ericphanson]

https://docs.github.com/en/pages/getting-started-with-github-pages/configuring-a-publishing-source-for-your-github-pages-site#publishing-with-a-custom-github-actions-workflow

It seems like pushing to a branch is not necessary, and instead a GitHub action can upload the files directly. This avoids repo bloat, and obviates the per-PR cleanup workflow, and likely also the workflow where docs are pushed to a separate repo. It would be nice if Documenter could support this method of using GitHub pages.

I think something like https://github.com/actions/upload-artifact/blob/main/merge/README.md would be necessary since each workflow run only generates the files for the current version, so there’ll be one artifact per run which then needs to be merged to be deployed.

I learned about this from https://github.com/JuliaRegistries/General/pull/120760#issuecomment-2526406007

[mortenpi]

I think something like https://github.com/actions/upload-artifact/blob/main/merge/README.md would be necessary since each workflow run only generates the files for the current version, so there’ll be one artifact per run which then needs to be merged to be deployed.

Yes, we need to merge with the previously deployed docs somehow. But if someone can demonstrate a working setup, I'd be very happy if we'd change the recommended workflow over to that. gh-pages is definitely not ideal.

However, while I didn't delve into it too much, but one issue with relying on artifacts is that they have a retention policy. So if you happen to not deploy for a while, you'd lose all the old versions I think?

[ericphanson]

ah, right, 90 days: https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/storing-and-sharing-data-from-a-workflow

I think that makes this idea untenable; we need persistent state and artifacts don't give that.

[vchuravy]

An thought I just had...

The main thing we want here is for documentation for tags to work. We can't rely on the persistence of Github Actions.

But tags do have a separate storage mechanism, "assets". Like we use for JLLs https://github.com/JuliaBinaryWrappers/GMP_jll.jl/releases/tag/GMP-v6.3.0%2B2

So a potential solution would be to build documentation on tag and upload the result as a tar'd assett. Then the deploy action can look through artifacts with a retention period (to support builds from release and main, as well as previews).

[mortenpi]

I think using release assets for storing the doc tarballs is conceptually the right thing to do, it's just more complex than the current approach (and not as universal). But I think it would be worth prototyping something for this.

Normally, the CI for the tag kicks off once TagBot creates the tag and the release, so it seems like it should be quite feasible to upload a tarball from the release build, and then trigger the deployment to Pages.

Then the deploy action can look through artifacts with a retention period (to support builds from release and main, as well as previews).

For what it's worth, I think it would also be okay to fall back to using a special branch for this (i.e. so that it would be retained forever). The main issue is gh-pages exploding in size due to many versions, but that would be much less of an issue if it just contains one version that gets replaced.

[quantumgc]

I've been looking into Documenter.jl for a separate project and noted that deploydocs already generates a tarball via a hidden flag (see here). Can this be leveraged in any way or is it a completely separate feature?

[mortenpi]

That flag is for the Julia manual build, where the CI jobs for building the docs and pushing them to a gh-pages branch are two separate jobs. So the docs needs to be uploaded as an artifact tarball, and then gets downloaded in the job that pushes to the Git repo.

I suppose, if you wanted to upload the tarball of gh-pages to GitHub Pages yourself (instead of letting GitHub package it up for you, as it's normally done right now), then you'd be welcome to use it. But you'd still have to also push to gh-pages too, so that the version that you just built would be available when you build the next one.