documentation icon indicating copy to clipboard operation
documentation copied to clipboard

Test the safe parts of notebooks that run on hardware in every CI run

Open Eric-Arellano opened this issue 4 months ago • 0 comments

We cannot easily test notebooks that run on hardware in CI for every PR because a) there is often a queue and b) it costs actual resources. So, we currently skip those notebooks in PR runs:

https://github.com/Qiskit/documentation/blob/8642507fc2c5fc4982e79f16f414ba66fc629708/scripts/nb-tester/test-notebook.py#L33-L43

We then run those notebooks every two weeks in a cron job with GH Actions.

However, we can actually run part of the notebooks safely! We can run every code cell up until the point when we call Estimator.run() and Sampler.run(), or do some other slow thing with Qiskit Runtime. All the code up to that point is safe to run.

We can add a Jupyter notebook tag like submits-job to the code cells that use hardware. Then teach nb-tester to only run up to that code block, but stop there. We wouldn't run any subsequent code blocks since they may depend on the tagged code block.

Eric-Arellano avatar Apr 12 '24 20:04 Eric-Arellano