[MRG] CI Cache

[chenghuzi]

Address #561

[jasmainak]

Is there a way to force a full build if there is a problem with the cache? This can sometimes be necessary

[chenghuzi]

Is there a way to force a full build if there is a problem with the cache? This can sometimes be necessary

The simplest way might be using a key to specify the cache

[codecov-commenter]

Codecov Report

Merging #573 (fc6008a) into master (1f550b1) will increase coverage by 0.04%. The diff coverage is 98.91%.

:exclamation: Current head fc6008a differs from pull request most recent head a6b3d4d. Consider uploading reports for the commit a6b3d4d to get more accurate results

@@            Coverage Diff             @@
##           master     #573      +/-   ##
==========================================
+ Coverage   91.69%   91.74%   +0.04%     
==========================================
  Files          21       22       +1     
  Lines        4204     4229      +25     
==========================================
+ Hits         3855     3880      +25     
  Misses        349      349              

Impacted Files	Coverage Δ
hnn_core/gui/gui.py	96.44% <ø> (+0.69%)	:arrow_up:
hnn_core/params_default.py	100.00% <ø> (ø)
hnn_core/cell.py	96.85% <95.65%> (-0.37%)	:arrow_down:
hnn_core/cell_response.py	84.00% <100.00%> (-0.54%)	:arrow_down:
hnn_core/dipole.py	92.45% <100.00%> (-0.05%)	:arrow_down:
hnn_core/gui/_viz_manager.py	88.36% <100.00%> (+0.90%)	:arrow_up:
hnn_core/network_builder.py	93.85% <100.00%> (+0.08%)	:arrow_up:
hnn_core/parallel_backends.py	81.11% <100.00%> (-1.12%)	:arrow_down:
hnn_core/viz.py	89.19% <100.00%> (+0.35%)	:arrow_up:
... and 6 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[chenghuzi]

Is there a way to force a full build if there is a problem with the cache? This can sometimes be necessary

The simplest way might be using a key to specify the cache

Or we can hash the source code folder

[jasmainak]

see: https://github.com/mne-tools/mne-python/blob/3b29980d22becbf8bfa1c535f6e5680ad4bc749a/tools/circleci_download.sh#L5

would be nice if you can add "[circleci full] commit message" and it does the full build ... or "[circleci gui] commit message" and it just rebuilds the GUI stuff ...

[chenghuzi]

see: https://github.com/mne-tools/mne-python/blob/3b29980d22becbf8bfa1c535f6e5680ad4bc749a/tools/circleci_download.sh#L5

would be nice if you can add "[circleci full] commit message" and it does the full build ... or "[circleci gui] commit message" and it just rebuilds the GUI stuff ...

Ah I see, this is better

[chenghuzi]

see: https://github.com/mne-tools/mne-python/blob/3b29980d22becbf8bfa1c535f6e5680ad4bc749a/tools/circleci_download.sh#L5 would be nice if you can add "[circleci full] commit message" and it does the full build ... or "[circleci gui] commit message" and it just rebuilds the GUI stuff ...

Ah I see, this is better

I found using environment variables might be even easier compared with this one... We just need to change the value of CACHE_VERSION in CircleCI project settings before re-run

[jasmainak]

But with cache-version, we'll have to go inside circleci and change it. And also, it can be done only by the owners of this repository. Personally, I prefer if you can just specify in commit message. Not sure how complicated it is to set up though ...

[chenghuzi]

But with cache-version, we'll have to go inside circleci and change it. And also, it can be done only by the owners of this repository. Personally, I prefer if you can just specify in commit message. Not sure how complicated it is to set up though ...

I prefer using commit messages too. But it seems CircleCI does not allow overwriting or clearing caches. If you try to just save_cache with an existing key, the output will be:

So it seems the only plausible way is to update the key name. Commit messages are stateless in this sense. If we want to manually update the cache without entering circle, we can use some key version file. Not sure if this is too ad-hoc...

Or we can export the git log file, to trace the last commit message in the current branch that contains full-build and use the corresponding hash as the key.

[jasmainak]

how is MNE doing this? did you take a look?

[chenghuzi]

But with cache-version, we'll have to go inside circleci and change it. And also, it can be done only by the owners of this repository. Personally, I prefer if you can just specify in commit message. Not sure how complicated it is to set up though ...

I prefer using commit messages too. But it seems CircleCI does not allow overwriting or clearing caches. If you try to just save_cache with an existing key, the output will be:

So it seems the only plausible way is to update the key name. Commit messages are stateless in this sense. If we want to manually update the cache without entering circle, we can use some key version file. Not sure if this is too ad-hoc...

Or we can export the git log file, to trace the last commit message in the current branch that contains full-build and use the corresponding hash as the key.

wait it seems that CircleCI does not respect the environment variable exported in the command for saving cache...

[jasmainak]

Thanks for your patience @chenghuzi ! It's merged now :)