Streamlining the deployment experience

[mortenpi]

As I was setting up Documenter for a package, the following thoughts crossed my mind:

• [x] It should be possible to add the GitHub deploy key automatically using GitHub's API. (see OnlinePackage.jl)
• ~In genkeys, travis asks for the password even if you are already logged in. We could check travis whoami to decide when to skip travis login.~ (genkeys has changed since)
• [x] The branch (e.g. gh-pages) should be created automatically if it doesn't exist (git checkout --orphan + cleanup). (#456)
• [ ] Would be nice if you could also easily deploy from a local machine. It shouldn't see that much use, but it would still be convenient for people who don't use or have disabled Travis, etc.
• [x] Also, documentation updates (#144).
• [ ] Check target directory somehow in deploydocs. Currently you can deploy nonsense docs successfully if target is wrong.

Travis UX

• [x] Be explicit about why deployment is getting skipped -- saying "Skipping deployment" is not very helpful to the user when things are going wrong and setting the DOCUMENTER_DEBUG variable is a bit inconvenient (#426).

• [x] Somehow notify the user when Documenter fails or when deprecated features are in use -- currently it is easy to miss that docs are not being updated and no way real for the user to notice any deprecation warnings Documenter might emit. Not exactly sure how this should be done though -- for failures we could try failing the build (would need a different default .travis.yml setup).

  Solution: Use Travis build stages, which is now the recommended way.

• [ ] Handle bad repo arguments better. The requirements for it were recently documented, but it is still easy to make a mistakes, and then the error Documenter gives is not very helpful.

[MichaelHatherly]

Yes, improving on the current experience would be good.

[MichaelHatherly]

Ref #144: for some extra discussion on this.

[tkelman]

Is the plan to deploy docs to a separate gh-pages or other branch at some point, it just hasn't been done yet? Many packages are starting to use this and generate docs on master under a docs folder, and it adds a fair amount of noise to reviewing code changes.

[mortenpi]

I'd say that Documenter is already designed to push to the gh-pages of the same repository. This is what all the packages I've seen do and what Documenter's manual recommends.

Do you mean that people are starting use the new feature where GitHub will host doc pages from master's docs/ directory? If so, then I think that should be discouraged and people should use gh-pages instead.

[tkelman]

Yes, I've seen several packages doing what looks like hosting html out of master.

[MichaelHatherly]

Yeah, using gh-pages is the only recommended way of hosting the generated docs. Committing generated docs, whether html or md, to master really shouldn't be done. As far as I can tell there aren't too many packages doing that. All the well-known packages with any significant number of stars are doing things correctly, so hopefully that is enough to encourage new packages to do things the right way.

[tkelman]

hopefully that is enough to encourage new packages to do things the right way.

It isn't, I'm seeing an increasing number of packages put their built docs in master. The only way people will do otherwise is if the documentation here explicitly tells them to not do that, I suspect.

[MichaelHatherly]

It isn't, I'm seeing an increasing number of packages put their built docs in master.

Probably partly due to GitHub's new "host your docs in docs/ on master"-feature, which is fine for simple markdown files, but not for what we're doing here. #325 should hopefully make it a little clearer early on in the guide that we don't support committing content to master and that it's frowned upon.

[mortenpi]

Added two things related to the automatic deployment on Travis:

• Be explicit about why deployment is getting skipped -- saying "Skipping deployment" is not very helpful to the user when things are going wrong and setting the DOCUMENTER_DEBUG variable is a bit inconvenient.
• Somehow notify the user when Documenter fails or when deprecated features are in use -- currently it is easy to miss that docs are not being updated and no way real for the user to notice any deprecation warnings Documenter might emit. Not exactly sure how this should be done though -- for failures we could try failing the build (would need a different default .travis.yml setup).

[tlnagy]

@mortenpi I'm running into this right now. Gadfly's docs are skipping deployment and I'm not sure why: https://travis-ci.org/GiovineItalia/Gadfly.jl/jobs/204402798

[mortenpi]

@mortenpi I'm running into this right now. Gadfly's docs are skipping deployment and I'm not sure why: https://travis-ci.org/GiovineItalia/Gadfly.jl/jobs/204402798

You're missing the DOCUMENTER_KEY variable, which is the only authentication method now. The deprecated ones were dropped in v0.9.0. Unfortunately we're not being very helpful with the errors regarding this at the moment. See the "Hosting Documentation" in the manual to see how to set up new keys.

[tlnagy]

I get

ERROR: Gadfly already has an ssh key. Remove it and try again.

~~How do I remove the key?~~ Gadfly/docs/.documenter.enc was the key that I had to remove.

[felixcremer]

The deployment of the docs can still fail silently because the deprecation of for example the julia keyword in deploydocs is handled via @info, which still exit the make.jl file with code 0 and therefore does not break the travis stage. See for example in LombScargle: https://travis-ci.org/JuliaAstro/LombScargle.jl/jobs/503879414#L528

[mortenpi]

@felixcremer I agree that it is in some ways suboptimal, but it is consistent with how the argument was handled before -- if it didn't agree with TRAVIS_JULIA_VERSION, it would silently skip deployment. As it is a handling of a deprecation and does not affect up to date setups, I don't think it's worth changing it now. At some point, when we remove the deprecated features, passing the julia keyword will become an error anyway.

[felixcremer]

I agree, that this is not worth changing now. I misinterpreted the failure as due to the deprecation warning. But this made me check an up to date Documenter build: https://travis-ci.org/felixcremer/EmpiricalModeDecomposition.jl/jobs/502557634#L608 This is working on Documenter 0.21.5 and the deployment fails because I misspelled the repo name in the repository url. I think it would be good to error out or exit with a non-zero exit code if the should_deploy variable is false. If you agree, I could prepare a PR.

[mortenpi]

I think it would be good to error out or exit with a non-zero exit code if the should_deploy variable is false.

deploydocs shouldn't error simply when it decides it should not deploy. E.g. if you run your builds for a PR, should_deploy will be false, but you'd still want the documentation build to pass.

[adigitoleo]

Would be nice if you could also easily deploy from a local machine. It shouldn't see that much use, but it would still be convenient for people who don't use or have disabled Travis, etc.

I'm playing with documenter at the moment and would love to have the option to build the ready-made site and just throw it in a folder on a separate local branch, without CI. I've been able to get something up using just makedocs, but I'm considering pulling out some of the HTMLWriter calls from deploydocs and adding them to my make.jl.

I'm a bit confused about makedocs, why doesn't it do the versioning etc.? Could that be decoupled from the git push stuff in deploydocs? Just curious, I've never really used CI before so maybe I'm missing something. I'm just not convinced that it's really worth it for small/niche packages (suddenly there are arcane yaml files and tokens all over the place). I guess most people already have something set up for running tests, but I can probably get away with running those using builds.sr.ht which is about the only CI I could understand at the moment :)

[mortenpi]

I'm a bit confused about makedocs, why doesn't it do the versioning etc.?

One answer would be that makedocs generally just produces a bunch of files for a single version of the documentation. Handling multiple versions is a layer above that, and also depends on stuff generated in earlier builds. The other answer is that, especially deploydocs' design has been very.. organic, i.e. things have been added to it as the need arises for some behaviour.

One option to improve on this would be to pull out the versioning stuff into a separate API that deploydocs then calls, but could also be called manually separately from it.

[adigitoleo]

I see. One more question, I can see that HTMLWriter.expand_versions looks for docs subfolders with the built docs for each version. Could you point me at where those folders are created? Or is that something that needs to be set up before building?

[mortenpi]

They are created by deploydocs when build/ gets copied over: https://github.com/JuliaDocs/Documenter.jl/blob/68dbd53d4ff6b339e795a4a3328955ad5c689e0e/src/Documenter.jl#L609

(with gitrm_copy defined here)