test-infra icon indicating copy to clipboard operation
test-infra copied to clipboard

Published 20 hours ago • verified publisher icon kubernetes

Allow pod-utils / clonerefs to use default remote branch

Open spiffxp opened this issue 4 years ago • 22 comments

My read of current pod-utils code is that it assumes base_ref to always be present / non-empty (ref: https://github.com/kubernetes/test-infra/pull/20665#issuecomment-769961333)

It looks like there are 435 jobs that explicitly reference master, when what they likely mean is "just use whatever the default remote branch is"

$ ag --no-filename "base_ref:.*master" config/jobs | grep base_ref | wc -l
     435

Ideally base_ref would be auto-populated with the remote default branch if omitted. Maybe this already happens? This would make repo default branch renames less toilsome.

Opening this to track verifying or fixing, so not yet a bug or feature.

/area prow/pod-utilities

/assign @alvaroaleman @cjwagner in case you can answer more quickly than I can, feel free to /unassign if you can't handle /assign @spiffxp to verify what happens today

spiffxp avatar Jan 29 '21 18:01 spiffxp

/sig testing

spiffxp avatar Jan 29 '21 18:01 spiffxp

My read of current pod-utils code is that it assumes base_ref to always be present / non-empty (ref: #20665 (comment))

Yeah, that is correct (for periodics, for presubmits/postubmits we get the info from github)

Ideally base_ref would be auto-populated with the remote default branch if omitted. Maybe this already happens? This would make repo default branch renames less toilsome.

I am pretty sure this does not happen. Adding this would be pretty easy from a complexity POV, as the default_branch is part of the repo. It will drive up token usage a bit though (and I think it won't cache that well, as the repo contains a lot of info that will regularly change, like pushed_at: timestamp, size, the number of stars, forks etc)

alvaroaleman avatar Jan 29 '21 19:01 alvaroaleman

If we're concerned about token usage, we could try doing this directly in clone by using the contents of .git/refs/remotes/origin/HEAD if base_ref is empty. I dunno if that makes the API feel too weird though (technically it does omitempty as-is)

spiffxp avatar Jan 29 '21 21:01 spiffxp

/area jobs since this would be a job config change too

spiffxp avatar Jan 29 '21 22:01 spiffxp

/wg naming since this is in support of https://github.com/kubernetes/org/issues/2222

spiffxp avatar Jan 29 '21 22:01 spiffxp

Relevant discussion thread in #prow https://kubernetes.slack.com/archives/CDECRSC5U/p1615236463055600

spiffxp avatar Mar 08 '21 22:03 spiffxp

Discussed at sig-testing meeting today:

  • Goal is to add support for this to extra_refs and not the spec for triggering presubmits/postsubmits
    • so, the extra_refs used by periodics
    • and also, any extra_refs used by presubmits/postsubmits that clone multiple repos
  • We might be able to use HEAD as part of the extra_refs spec today
  • If not, I'll try prototyping something, if it gets too complex, I'll pull back to a proposal for review

spiffxp avatar Mar 09 '21 18:03 spiffxp

Since the HEAD symbolic ref is not a default git thing and instead a real symbolic ref that e.g. GitHub, Gerrit, GitLab provide, it should be possible to just use extra_refs[*].base_ref: HEAD.

stevekuznetsov avatar Mar 09 '21 21:03 stevekuznetsov

FWIW, I feel like remotes having a HEAD is a default git thing, but maybe I'm wrong. I'll give the base_ref: HEAD thing a shot this week

refs:

  • https://git-scm.com/book/en/v2/Git-Basics-Working-with-Remotes
  • http://git.661346.n2.nabble.com/What-s-the-definition-of-a-valid-Git-symbolic-reference-tt6025154.html#a6026446

spiffxp avatar Mar 09 '21 22:03 spiffxp

not quite workable, because we try to check it out to a branch with that name, and it's not a valid branch name https://github.com/kubernetes/test-infra/pull/21452#issuecomment-803075138

BenTheElder avatar Mar 19 '21 20:03 BenTheElder

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

fejta-bot avatar Jun 17 '21 20:06 fejta-bot

I think we could add something simple to replace HEAD with a valid branch name and should still consider this.

BenTheElder avatar Jun 17 '21 20:06 BenTheElder

/remove-lifecycle stale /milestone v1.23

spiffxp avatar Jul 13 '21 16:07 spiffxp

/kind feature

spiffxp avatar Oct 01 '21 19:10 spiffxp 

$ ag --no-filename "base_ref:.*master" config/jobs | grep base_ref | wc -l
    1141

The number of jobs affected has grown considerably from the 435 that existed at the beginning of this year

spiffxp avatar Nov 24 '21 01:11 spiffxp

/milestone v1.24

spiffxp avatar Nov 24 '21 01:11 spiffxp

/priority important-longterm

spiffxp avatar Nov 24 '21 01:11 spiffxp

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Feb 22 '22 01:02 k8s-triage-robot

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot avatar Mar 24 '22 02:03 k8s-triage-robot

@cjwagner @stevekuznetsov @chaodaiG WDYT https://github.com/kubernetes/test-infra/issues/20667#issuecomment-863556005 ?

I think we could do this pretty easily by just detecting HEAD => pick a different value for the branch name but otherwise use HEAD in most places untouched. Last time I tried this the branch name seemed to be the only stumbling block. I'm not sure what we should call it.

BenTheElder avatar Apr 19 '22 02:04 BenTheElder

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

  • After 90d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
  • After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

  • Mark this issue or PR as fresh with /remove-lifecycle stale
  • Mark this issue or PR as rotten with /lifecycle rotten
  • Close this issue or PR with /close
  • Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot avatar Jul 18 '22 03:07 k8s-triage-robot