No automatic updates if GIT_TAG points to a branch

[DmitryYurov]

I am trying to use cpm for automating dependencies of my projects. Unfortunately if I use cpm like

CPMAddPackage(
  NAME name_of_repo
  GIT_REPOSITORY ssh://git@server/project.git
  GIT_TAG master
  EXCLUDE_FROM_ALL YES
)

the fetched resource does not track the master branch - that is, on update of my master branch I still get older commit in the local repository. On the other hand, if I manually remove the fetched data, new download looks at the proper commit.

If that matters, I set CPM_SOURCE_CACHE environment variable. Is it a bug or expected behaviour? I would expect that resources are downloaded again if the branch state in remote repository changes.

[TheLartians]

This is expected behaviour, as CPM.cmake assumes that the TAG argument point to an unchangeable git tag or commit hash. We encourage this, as otherwise builds would become unstable, e.g. they would change every time the branch is modified. Also this allows us to efficiently cache sources and have a predictable offline and online behaviour.

For parallel development of dependency / dependant projects, I would either manually update the tag to the most recent commit or use a local package override to pull the source from a local directory that is kept up to date with the master branch.

[DmitryYurov]

In this case the functionality related to branch-named tag seems to be internally contradictory - on the one hand if I reference the tip of the branch, I would like the reference to stay on this tip - but it is fixed on the last commit at the moment of the first dependency download. On the other hand, this tag does not provide the stability, since the user could for example setup a clean build on the server and in such a way update the reference all the time. The behavior of fixed dependency in case of a version tag looks good, but with branch-named tag it is quite unexpected.

[TheLartians]

You're right, the README is actually wrong in this case as it was written before we implemented caching and offline support. We actively discourage the use of branch names, as it would prevent us from supporting offline builds and reproducibility due to the reasons mentioned above. Perhaps we could add an alternative option BRANCH_NAME that would allow the old functionality of auto-updating branches.

[N3m3si5]

You're right, the README is actually wrong in this case as it was written before we implemented caching and offline support. We actively discourage the use of branch names, as it would prevent us from supporting offline builds and reproducibility due to the reasons mentioned above. Perhaps we could add an alternative option BRANCH_NAME that would allow the old functionality of auto-updating branches.

googletest recommends to use latest master: see Live at Head. Therefore I personally think this is a handy feature in some circumstances. 👍

[anb0s]

I've tried CPM and it works very well! Thank You for sharing! (imho: it could be official part of the CMake distribution ;)

The only issue is to support work with multiple dependencies as local repositories "in work" and referencing therefore latest branch commits for "integrators". Here we want to have automatic updates from latest branches and set many tags as workaround does not feel right ;) During the merge to mainline we verify that only tags or commits are used in such "read only" repositories (dependencies) for all other developers.

Do you have plans to introduce BRANCH_NAME to support this behaviour?

[TheLartians]

Thanks for the kind feedback, glad it's useful for you! :)

Do you have plans to introduce BRANCH_NAME to support this behaviour?

Currently no plans, though what speaks against just using plain old FetchContent in this case? I believe it should match the expected behaviour for branches - including loosing most advantages of CPM.cmake anyways (no caching, no offline builds, slow to configure etc).

[anb0s]

Thanks for the kind feedback, glad it's useful for you! :)

Do you have plans to introduce BRANCH_NAME to support this behaviour?

Currently no plans, though what speaks against just using plain old FetchContent in this case? I believe it should match the expected behaviour for branches - including loosing most advantages of CPM.cmake anyways (no caching, no offline builds, slow to configure etc).

Yes, sure, that's how we handle this - we are using submodules or FetchContent. I just tried to have CPM as the dependency manager in all places independent how the dependency is consumed (remote location etc.) and checked out like local path (sub-dir) in source tree or ouside source tree e.g. in the build folder. At the end i want to have simple clone without dependencies (no submodules etc.) and CMake configure should handle all dependencies via CPM :)

• external dependencies (read-only): FetchContent in the build folder and build
• internal depedencies (possible for contributions): FetchContent or native Git as submodules in source tree

[DNKpp]

If I understand you correctly, your main reason against the auto updating is, that this doesn't perform well in an offline environment. That's completely understandable but I wonder if a step-in-the-middle could help. If one target a branch, why not simply try making a connection and see if the origin has been updated. If no connection is possible, than simply use the local version. That's what I expect if I use a cache. Because I would really like to use a environment wide CPM_SOURCE_CACHE on my laptop, a fallback to FechtContent is not possible for me. Manually deleting all of the downloaded deps doesn't feel right.

EDIT: Perhaps a CPM_PREFER_UPDATE option for CPMAddPackage would already fit the needs. If I think about huge projects like boost or protobuf I would rather strictly stick to a fixed version (thus explicitly targeting a specific tag or commit hash), but for small frequently updating libs (for example entt oder catch2) I would like to keep up2date without manually forcing the update.

[OlivierLDff]

From experience I wouldn't recommand using git branch, because then it's hard to track in the life of the project which dependency is expected where. Also, if a branch name change, or is deleted, then you are just left with an error. All of my old commit won't compile with googletest because they change master to main. So each time I go to an old commit I need to fix that. I would have used git tag back then I would be fine.

Lesson learned:

• Never use public github repository, always fork it to be sure what you have in there (and then you can maintain branch if you want). That will also help avoid malicious commit, deleted branch, repo deleted by user, etc...)
• Use git tag when you are in control of the repository (ie you forked it). So that you can clearly see in your commit history the dependency that evolve.
• Or use commit sha, if you are develloping the dependency library at the same time.

Expecting auto update will only lead to commit not compiling in the future, and making tools like git-bisect ineffective.

Edit: auto update should be handled by a tool like dependabot

[TheLavaBlock]

Hey, in my library liblava I integrate external modules rolling release-like, so I solved it with a simple python script update.py which updates all GIT_TAG when the opportunity arises. If you are interested, please take a look. It would be great if there was such a mode in CPM.cmake and it all worked automatically.

[TheLartians]

@TheLavaBlock that's a clever solution, thanks for sharing! I completely agree that this would be a great feature to have built into CPM.cmake itself, but the implementation would be a bit tricky due to edge cases and the imperative nature of CMake itself.

Note that we have a related discussion on this feature here.