buildkit
buildkit copied to clipboard
Proposal: csv syntax for git repos
Today's way of expressing remote Git repos as contexts
https://github.com/user/repo.git
https://github.com/user/repo.git#
https://github.com/user/repo.git#:
https://github.com/user/repo.git#:dir
https://github.com/user/repo.git#branch_or_tag_or_commit
https://github.com/user/repo.git#branch_or_tag_or_commit:dir
does not allow
- specify both a commit hash and e.g. a branch (as image digests do:
$IMAGE_URL:mytag@sha256:cafebabe..
)-
Note importantly: tools such as
renovate
provide a way to upgrade digests givenmytag
-
Note importantly: tools such as
- differentiating between a tag or a branch (and a non-full commit)
To permit this I propose to extend this syntax in a somewhat backwards compatible way:
https://github.com/user/repo.git##tag=mytag,branch=main,commit=cafebab,source=/dir
-
tag
andbranch
cannot be used together -
commit
can be a short commit hash instead of a full one - note the leading
/
forsource
: to keep with uniformity with other csv args -
##
usage (instead of just#
) may already exist in the wild (but "unlikely" combined with csv) -
renovate
can work withbranch
ortag
(git tags are as mutable as branches) andcommit
, as with images
differentiating between a tag or a branch (and a non-full commit)
FWIW, the simpler answer to this is to use full/explicit refs instead of abbreviated ones (refs/tags/xxx
vs refs/heads/xxx
or even GitHub's fancy refs/pull/xxx/head
refs for test-building pull requests, for example).
(IMO [not a buildkit maintainer], it would be weird to invent buildkit-specific syntax sugar for these refs such that tag=
gets translated to refs/tags/
and branch=
to refs/heads/
)
tag and branch cannot be used together
As @tianon said this can already be done with setting full ref. Additionally, I think builds from refs like refs/pull/xxx/head
are very useful, so if we added a new syntax that capability would need to be kept. Even with CSV I would prefer ref=..
instead of branch=,tag=
.
commit can be a short commit hash instead of a full one
This looks like too much sugar to me. You can't do docker pull alpine@sha256:partialhex
.
renovate can work with branch or tag (git tags are as mutable as branches) and commit, as with images
Don't exactly get what case you have in mind here.
Downsides:
- CSV is quite cumbersome to read and write.
##
looks weird as well.
Potential opportunities with CSV format:
- Ref and commit can't be used together atm in a URL. That is important use case for replaying a historic build from a git ref, with a pin to a specific commit (this info is provided by provenance attestation). Another question is what
ref=,commit=
combination actually means, and I guess it should just ignore/trust the ref in that case and use the commit. Ref might still be visible as current branch in.git
. This is similar to how tag is ignored on container image refs if digest is present. - Keeping/discarding
.git
directory could be controlled by a property that is more flexible solution than currentBUILDKIT_CONTEXT_KEEP_GIT_DIR
build arg. - If keeping
.git
then another property could be used to control the depth of the cloned commit chain. - In https://github.com/moby/buildkit/pull/4646 , subdir component can potentially split into subdir + filter components.
Agreeing on all points.
Note about renovate
(and the majority of my motivation here): is to have a ref=..,commit=..
pair for git repos as we have $IMG:..@sha256:..
for images. That is, a constraint + a lock. Tooling can use this to auto-upgrade code.
I guess it should just ignore/trust the ref in that case
I understand this is today's behavior WRT image tags. I would argue that some kind of check that ensures that the given digest exists in/among/as the given constraint (at least once, some time in the past). e.g. make sure $commit exists on $branch.
I remember a beforetime when GitHub would be fine showing commits from some fork as commits of the upstream, trusted repo.