CTSM icon indicating copy to clipboard operation
CTSM copied to clipboard

Break up "omnibus docs test"

Open samsrabin opened this issue 6 months ago • 2 comments

Right now doc/testing.sh is scheduled to run in its entirety whenever the docs are updated, via docs-omnibus.yml. This should be broken up for clarity and to minimize the number of GitHub workflow runs that need to happen.

As a first step, each part of the script itself should be broken up into component scripts. This will enable us to keep testing.sh around (it's useful for local testing) while also breaking it up into different workflow jobs.

Second, we should exclude changes to testing.sh from triggering docs-related workflows.

Third, the "Build all docs using container" test should be added to docs.yml. It should run regardless of whether the existing build-docs job passes, because theoretically it's possible for either to pass while the other fails.

Finally, docs-omnibus.yml should be broken up into two workflows with the following tests from testing.sh:

  1. docs-updated-doc-builder.yml
  2. docs-updated_ctsm_pylib.yml

These would run only when their respective parts are changed. Both would contain these three parts, which constitute one test (job):

  • "Build all docs using container"
  • "Build all docs using ctsm_pylib"
  • "Make sure container version is identical to no-container version"

They would also both contain a job for "Check that Makefile method works," since it depends on both docs-builder and ctsm_pylib.

docs-updated-doc-builder.yml would additionally contain the "Check that -r -v works" test. The "Check that -b works" test would be excluded from new GitHub workflows as it's already part of docs.yml.

samsrabin avatar Jun 02 '25 17:06 samsrabin

This is a really good idea and good goals, allow for fewer workflows to trigger and break things up for clarity. Glad you'll work on this.

ekluzek avatar Jun 02 '25 20:06 ekluzek

From @ekluzek here, re: the ../bin/git-fleximod update -o line in testing.sh:

This makes sure that the optional submodules are checked out. Since, doc-builder is needed it's good to make sure it's there.

The caveat to this though is that it will do this as part of running testing.sh could potentially update other submodules and the user could be unaware of that. Since this script is mostly run by git-workflows that might be fine. But, as a developer I don't like running a script that ends up modifying my sandbox. As such I'd prefer if this step were outside the script. But, the script could/should make sure that doc-builder is there and updated appropriately.

samsrabin avatar Jun 05 '25 19:06 samsrabin