Ribasim icon indicating copy to clipboard operation
Ribasim copied to clipboard

Try QuartoNotebookRunner.jl to speed up docs build

Open visr opened this issue 9 months ago • 1 comments

Currently our docs build take 26 minutes. Our example notebooks take up a lot of that time. They are executed in Python, and then run the core with subprocess starting julia for each model. So it needs to load and compile each time. I suspect that if we run from Julia instead it will be faster since compiled code remains available in the session. With QuartoNotebookRunner v0.13 it should be possible to combine both Python and Julia.

https://discourse.julialang.org/t/ann-native-julia-engine-for-quarto-using-quartonotebookrunner-jl/112753/56

visr avatar Feb 18 '25 22:02 visr

I made good progress on this in https://github.com/Deltares/Ribasim/compare/main...qmd but one major downside to this is that we'd have to start worrying about compatibility between the Julia and Python ecosystem for shared dependencies. If a conda environment is not already present, CondaPkg forces OpenSSL to version 3.0 LTS to ensure compatibility. If I do that to our pixi environment, we downgrade many packages since Conda doesn't use the LTS release. For instance we'd downgrade Python to 3.11, NumPy to 1.26. EDIT: now that Julia uses the new 3.5 LTS this shouldn't be an issue for now.

https://github.com/JuliaPy/CondaPkg.jl/issues/172

Any work on #1942 will also alleviate the doc builds. If needed we could do something fancy like https://github.com/dmolina/DaemonMode.jl but I'm not sure it's worth the trouble, so closing this for now.

visr avatar Mar 11 '25 15:03 visr