hnn-core
hnn-core copied to clipboard
jonescompneurolab
CircleCI jobs speedup
Now it takes about 25 mins to build our doc and most of the time is spent on the doc building step.
And a large part of dock building is about running examples. Could we use some caching mechanism to avoid running them repeatedly if the corresponding code is not changed?
yeah that's something we could certainly consider ... caching can make debugging harder though. Are you still running the notebook twice? I think you mentioned that nbconvert doesn't use "run all", so you could use nbconvert once and avoid running the tutorial twice?
yeah that's something we could certainly consider ... caching can make debugging harder though. Are you still running the notebook twice? I think you mentioned that nbconvert doesn't use "run all", so you could use nbconvert once and avoid running the tutorial twice?
No it's not running twice anymore. And even with double running, the time is mainly spent on previous examples as I noticed that some simulations run >=300ms so they're really slow
@chenghuzi take a look here: https://github.com/mne-tools/mne-python/blob/main/.circleci/config.yml
I think $ make html with sphinx-gallery should only rebuild the examples that have changed if you cache them
I was about to close this issue, however, we seem to be having a problem getting CircleCI to pass on master. Any thoughts @chenghuzi?
yes I think we can safely close this now. For the CI problem, which one are you referring to? the MPI one?
No, it has something to do with the document build process for the GUI timing out. See here.
I fetched the corresponding PR and it works locally. I suspect it's the cache that failed the building process. I'll dig further to see if it's really the issue.
Wait I got it succeed here. This might due to the different namespace for the same cached content. I'll re-run to see if it works
Are you sure it's a cache issue? I suspect it succeeds stochastically depending on the CI load. You might need to make the example more verbose to stop that error for good.
Are you sure it's a cache issue? I suspect it succeeds stochastically depending on the CI load. You might need to make the example more verbose to stop that error for good.
I'm not sure as both my local build and a new build in my repo on CircleCI worked. On the other hand, I tried to re-run your build and it failed again, which made me to guess it might be the cache key. You can check if it's the case simply by changing the content in .circleci/build_cache.