Cache opam install --deps-only

[aantron]

Is there a plan to cache the opam dependency builds?

[dra27]

#63 facilitated this for v1. It's possible, but it has a risk of being unstable - in #63 I anticipated setup-ocaml facilitating caching of dependencies rather than actually doing it. There are two problems:

• All relevant opam files need to be included in the cache key. While one might hope that it doesn't happen frequently, that then means you can end up with multiple opam caches with multiple copies of the same compiler, which becomes a cache space concern very quickly (in #63, I did some crazy differential caching over the base opam switch). A simpler mitigation for this would be promoting the compiler to be ocaml-system-like, but that comes with its own complexities too!
• To have this fully automated, we'd really want ocaml-ci's analysis phase to blow away the cache when the dependency solution changes (again, in #63 I weakly left that to the user).

Short-term, the hope is that the Dune-cache should mitigate dependency rebuilds - although I completely agree that restoring a cache is still faster than running a build system, even if all the artefacts are cached!

[aantron]

I will look into this eventually a bit. With my superficial knowledge of GHA, I don't understand what could realistically be blocking this for non-Windows systems. By comparisons, I routinely cached both .opam and _build in Travis, achieving builds about one minute long for most rows. GitHub Actions so far is considerably behind this.

One of the main advantages of Travis was that it did not require coming up with cache keys. It did its own automatic scan of whatever directory you asked to cache. All that was required was opam clean.

[aantron]

(The rest of Travis' impllicit cache key consisted of all the environment variables set for each build row by the matrix)

[dra27]

All contributions very much welcomed, obviously 🙂

I expect your Travis experience - with all due respect to your projects - is because the builds were simple. My experience maintaining both opam and Dune's Travis scripts was that the caching was anything but simple. Travis has a serious UI advantage over GHA at the moment, though - if anything went wrong with the cache in Travis, there was a simply UI to delete the cache. Assuming you were doing opam update in the workflow, the cache would have been updating on many of the runs, I imagine? That was fine in Travis which imposed virtually no limits on caches, but the space available for GHA caches would be blown away by effectively having a full opam cache per PR.

[aantron]

Assuming you were doing opam update in the workflow, the cache would have been updating on many of the runs, I imagine?

I was not doing opam update in the workflow, unless there was no cache. I took effort to avoid any cache updates the vast majority of the time. Separate scheduled ("cron") jobs ran monthly to repopulate the cache with fresh dependencies.

My experience maintaining both opam and Dune's Travis scripts was that the caching was anything but simple.

I would be interested in details (beyond any Windows issues) if we will compare workflows and builds.

In any case, delaying caching for all users due to difficulties with caching a few complex users is something I personally would not choose to do.

[aantron]

I have to say I'm puzzled that you would assume someone would deliberately do opam update while also doing caching. Doesn't the former defeat the point of the latter? Quite the opposite, I spent several days tucking in all loose ends to get Travis to recognize that there was no cache change across many builds. That includes not caching new intermediate artifacts in _build, as well. My caching setup would load _build from cache if available, but not cache the updated _build afterwards. It would basically have a _build snapshot generated by the monthly cron job, and diverge from it gradually within that month, without updating the cached _build, until the next month's cron job re-created a fresh _build.

I also set up fast caching for a BuckleScript and esy build of some binaries, which would then be committed back into the repo into a branch for testing, which was again a cached build (of some of the other stuff involved in the testing).

There were many other wrinkles.

Given all that, I think it's best not to assume something about how simple the setup was, though I am sure both Dune and opam have their own interesting difficulties.

[avsm]

Do you have a link to your Travis setup? It's hard for me to form a view on the complexity of the technicalities without something specific to compare against.

[aantron]

https://github.com/aantron/bisect_ppx/blob/b266182961be6ae16fa38f7d5d1fc0d62e31644d/.travis.yml

[aantron]

Part of the setup is in Travis' own UI, but that just consists of setting up the "cron" job to run each month and providing a secret for commiting into the repo.

[aantron]

The setup is also running some expect tests conditionally on compiler version, and doing other such things like submitting to coveralls only conditionally, and doing a recursive self-test only conditionally. Some of the details of the involved commands are found in the Makefile, but they are not directly relevant to the caching of the most expensive parts, .opam and _build.

[dra27]

I think you threw me a curve ball by using the word "routinely" - that beautifully tuned Travis pipeline looks anything but routine to me 😀

I don't think that anything in this action precludes manually tuning the caching as you were before? setup-ocaml has taken the view that features should only be rolled out across all three platforms simultaneously, or not at all.

[aantron]

Fair enough :) I was interpreting routinely in a different way when I wrote it, which is that similar setups work across several projects for years without problems, but I can see how "routinely" could be interpreted as very wide adoption, and the Travis equivalent of using the older published Travis setup for OCaml.

I guess I was effectively being sneaky trying to switch to its GHA spiritual successor from a bunch of custom setups :)

Yes, I can try to work this out fully manually in GHA. If I get the time to do it, I will report on anything that may be of interest to setup-ocaml.

We do plan to support Windows in Dream, which is the project with the GHA setup that triggered this issue, so I may even find something that can work on all platforms.

[dra27]

Travis caching definitely has one over GHA caching with the automatic detection of changes (as you said in https://github.com/ocaml/setup-ocaml/issues/159#issuecomment-882409583) - the need to generate the cache key before retrieving things is less intuitive/general than the Travis approach was (even if GHA's is technically more principled, and does allow fall-back caches).

On the ocaml-ci analysis phase I mentioned earlier (that's this kind of step determining the exact set of opam packages which will be installed), we're looking at providing a GHA version of that based on setup-ocaml - the solution derived in the analysis phase gives a clear cache key for storing the dependencies, and in doing that we might be able to see how to feed that back into setup-ocaml itself.

[Kakadu]

I understand that caching opam-installed packages could be tricky. But if this feature is going to be implemented, it will be great

[Termina1]

+1 to this feature

[smorimoto]

Not only does the dune-cache handle this in a not so bad way, it also has the functionality of automatically sizing the cache to avoid cache thrashing and attempting to land it at the most reasonable intersection. It works well enough for my team, but is it still lacking something?