copr icon indicating copy to clipboard operation
copr copied to clipboard

Add support for snapshots

Open msrb opened this issue 1 year ago • 2 comments

Use case

I have an upstream code repository and I have CI configured for this repository. Every time a new commit is pushed to the repository, a new COPR build is triggered. I have a container image which, among other things, installs the RPMs from the COPR repository -- i.e. the COPR repo is a dependency of this other containerized project. I want to be able to update the COPR RPMs in the container to a specific version, and only when I decide to do so.

The problem

I cannot easily pin down versions of RPMs in the COPR repository.

  • The COPR repo closely follows upstream (it's a build of the main branch)
  • Every time I rebuild the image, I get the latest COPR build (even when I don't want to -- maybe I am simply rebuilding the image because I am updating other things)
  • I cannot pin down a specific version of the COPR RPM, because COPR garbage collects older versions -- my image build would start failing

Workaround(s)

  • I can create 2 COPR repositories -- one for the latest-greatest version that follows the main branch, and the second repo where I rebuild (!) a specific version that I want to be using in the container
  • I can fork the main COPR repo every time I want to pin down a specific version and then point the container build to this forked repo

Proposal

Make it possible to create snapshots of COPR repositories. And make it possible to enable COPR repositories (via dnf) and use a specific snapshot (or the "main", if no snapshot was specified). I imagine that the workflow would look like this:

  • I have one COPR repository which follows the upstream code repo
  • I can decide to create a snapshot of the current state of the COPR repository
  • I update the dnf copr enable ... snapshot reference in the Dockerfile
  • I test that the container image works as expected and that the updated COPR app integrates well with the rest
  • I can change other scripts/apps in the container without updating the COPR app, i.e. I decide when the app gets updated
  • I can revert to the previous snapshot, if needed (because snapshots wouldn't be garbage collected)

msrb avatar Jun 13 '24 15:06 msrb

Thank you for the RFE @msrb.

I can fork the main COPR repo every time I want to pin down a specific version and then point the container build to this forked repo

This is an intended usage of the forking feature. Is there a drawback for you, making this to be a workaround rather than a solution?

and the second repo where I rebuild (!) a specific version

Depends on how many and how big packages you have in the repositories. But if it is something reasonable, rebuilding them should be fine as well. Copr's builder farm is quite large.

Workaround(s)

There are two more workarounds that could maybe work for you.

  1. Creating repositories manually - https://docs.pagure.org/copr.copr/user_documentation.html#creating-repositories-manually
  2. CoprDirs - AFAIK they are not well documented for the end-users. But you can see them in action - https://copr.fedorainfracloud.org/coprs/g/copr/copr-pull-requests/builds/ here. Above the builds table, there are buttons for switching between the individual "CoprDirs". Builds in each CoprDir are sandboxed and don't see builds from another CoprDir and also the garbage collection you mentioned should consider every CoprDir separately. There is no mechanism for moving the builds from a CoprDir to the main project repository (maybe there should be?), you will have to resubmit the build to do that. But this feature will help you keep everything within one project. One more thing, you can freely create CoprDirs using copr-cli, the only limitation is that they have to start with :custom: prefix.

Make it possible to create snapshots of COPR repositories.

Personally, I don't think we will have the resources to implement the snapshotting feature but the CoprDirs feature might be quite close? Documenting it and making some smaller improvements to it would probably be a better approach for us.

Somebody will comment with more information after a team meeting.

FrostyX avatar Jun 19 '24 11:06 FrostyX

We will delay it until we have Pulp as the backend (that alone should happen this year). With Pulp, it should be kinda easier to do. But still not big priority for us.

xsuchy avatar Jun 19 '24 11:06 xsuchy