pastas
Uncertainty NBs not run on RTD due to timeouts
Some of the notebooks, particularly those interested in uncertainty estimation, take too long to run on RTD and are then stopped with a timeout warning (I think set to 30 seconds per default).
Since RTD is already running for a long time, the solution should not be to increase the timeout time, but to
a) Update the example to be faster b) Update the code to be faster (perhaps parallel?) c) Not run the NBs on RTD anymore, but perhaps through a separate GH action, since they are already run somewhere as well anyway. And use that NB output as input for RTD. Don't know I this is possible but that would be interesting.
The new notebook is now listed and can be found here: https://pastas--903.org.readthedocs.build/903/examples/uncertainty_ls_mcmc.html
It suffers the same problem as the other notebooks that take too long to run. I think for now however we should accept that because the validity of the results depend on sufficient steps. For the other MCMC notebook this is less important because it only shows how it can be used without checking the results.
Originally posted by @raoulcollenteur in https://github.com/pastas/pastas/issues/903#issuecomment-2700744430
b) Update the code to be faster (perhaps parallel?)
A pure pastas solution is still in the works of course, but you could use pastastore if you're working with multiple models. This example notebook shows how you can get prediction intervals in parallel.
c) Not run the NBs on RTD anymore, but perhaps through a separate GH action, since they are already run somewhere as well anyway. And use that NB output as input for RTD. Don't know I this is possible but that would be interesting.
This is possible, flopy does this for example. The completed notebooks are uploaded as an artifact that is downloaded by RTD and converted to documentation. It does require some extra configuration though, which can be annoying to get right.
Cool, sounds like option (c) is the way to go then! It's probably also environmentally moroe friendly to not run the notebooks in multiple locations each time :D
I'll have a look if I find time within the NWO project over the next couple of months.
We could also consider cancelling all running tests when one of them fails and cancelling the old test when a new commit is made. This will also reduce the compute costs.
c) Not run the NBs on RTD anymore, but perhaps through a separate GH action, since they are already run somewhere as well anyway. And use that NB output as input for RTD. Don't know I this is possible but that would be interesting.
@dbrakenhoff and I once did an attempt to implement this on the rtd_ghactions branch
We could also consider moving the documentation website from readthedocs to GitHub pages. This means that instead of pastas.readthedocs.io the documentation will be hosted on pastas.github.io/pastas. The GitHub pages is created with a GitHub action, which is beneficial for local testing. Additionally, there are no advertisements from readthedocs on the documentation website any more.
I will look into this and try to implement option C over the next couple of weeks.
Also worth considering,
- run them in parallel using
pytest-xdist. GH Actions has 2 cores, not sure what types of machines RTD has?