traits icon indicating copy to clipboard operation
traits copied to clipboard

Add documentation build to weekly cron job

Open mdickinson opened this issue 3 years ago • 1 comments

The CI issue with Sphinx 4.0 (#1463) went undiscovered for several weeks, only to rear its head when we wanted to make a PR for something unrelated.

We many want to consider adding the documentation build to the weekly cron job. Similarly, it would make sense to add style checks to the weekly cron job, in case an upstream change to flake8 breaks those checks.

Separately from that, we should consider pinning the versions of external packages used in the CI testing.

mdickinson avatar May 14 '21 08:05 mdickinson

As motivation, the twin goals here are:

  • No CI surprises. The CI should be reasonably reproducible. If I make a PR this week from my branch and the CI passes for that PR, then the same should happen if I make the same exact PR from the same code next week. Or: if the CI on my PR is passing and I make a trivial change (a typo fix in a comment), the CI should continue to pass regardless of any upstream package changes (Sphinx, flake8, ...).
  • Advance warning of incompatibilities with other projects. If the latest version of Sphinx breaks our documentation build, we want to know about that sooner rather than later, but without that getting in the way of other development work.

We may want three different sets of CI runs:

  • The CI run executed for a commit to a PR; this should use pinned versions of dependent packages to maximise reproducibility.
  • The exact same CI run as above, but executed on a cron job. This will save us from situations where we resume Traits development after some weeks of inactivity only to find out that the CI is broken on master. If this fails, fixing it should be an urgent / high priority action.
  • A cron job CI run that runs with the latest packages available from PyPI, to serve as advance warning of potential incompatibilities.

mdickinson avatar May 14 '21 08:05 mdickinson