Source file cache

[BHASVIC-JasperShovelton22]

Every time the local worker is started for a new batch of games, it must pull the sources of both repos and compile. However, often the same repos are pulled repeatedly, with only a tiny difference between the commits (or even no difference, in the case of official-stockfish/Stockfish). Should there be an option for a caching system? There could be a directory of downloaded repos, for which building would only require a pull from github and recompilation of 1-2 object files. Maybe the name for each pulled repo could be [github username]-[branch]-[SHA of latest commit]. The option on the worker might be:

-s {False,True], --use-cache {False, True}
                        cache downloaded repositories (default: False)

This would only save a few minutes at a time, but it would be nice to skip most of the 1-2 minute stage of building the sources. EDIT: crap, wrong github account. Oh well, too late.

[Sopel97]

How many % of the time of the worker is spent on compilation.

[vondele]

also, we do have a cache of binaries, so the same sources should be pulled only once from github.

[vdbergh]

Well the OP is mainly saying that only the patched files should be recompiled and not the whole source. But this is not how Stockfish is compiled if you do a profile build.

[vondele]

I see. Right, no that we shouldn't attempt. As indeed this doesn't really help with profile-build and LTO.