documentation icon indicating copy to clipboard operation
documentation copied to clipboard

Dependency upgrades should always run every notebook, not only changed notebooks

Open Eric-Arellano opened this issue 1 year ago • 0 comments

https://github.com/Qiskit/documentation/pull/1831 now executes all notebooks when requirements are changed. That's a big improvement so that we know we aren't breaking notebooks unintentionally when upgrading dependencies.

However, there's one edge case that can be deceptive: as soon as you change one of the notebooks, we stop testing every notebook and instead test only the changed notebook. Imagine this scenario: 5 notebooks have issues with the upgrade, and the author fixes one of them. CI will stop running on those 4 notebooks, so they will look okay even though they are not.

If scripts/nb-tester has changed, we should always run on every notebook.

Eric-Arellano avatar Aug 16 '24 17:08 Eric-Arellano