git-client-plugin icon indicating copy to clipboard operation
git-client-plugin copied to clipboard

[JENKINS-55875] Forcibly update git submodules

Open NuclearCookie opened this issue 5 years ago • 23 comments

JENKINS-55875 - Forcibly update git submodules

Currently, submodules are not updated with --force. This means if there is a local change, then the checkout fails, causing the build to fail.

This can be worked round by cleaning the repository before every checkout, but this makes build jobs much slower, as they aren't able to be incrementally built, as the entire workspace has to be thrown away each time, or requires manual intervention.

Instead, this changes the checkout to be done forcefully, which matches how the checkout of the top level Git repository works.

This change should not have any harmful effects - it will only cause builds which were failing to now succeed.

Checklist

  • [x] I have read the CONTRIBUTING doc
  • [x] I have referenced the Jira issue related to my changes in one or more commit messages
  • [x] I have added tests that verify my changes
  • [x] Unit tests pass locally with my changes
  • [x] I have added documentation as necessary
  • [x] No Javadoc warnings were introduced with my changes
  • [x] No spotbugs warnings were introduced with my changes
  • [ ] I have interactively tested my changes

Types of changes

  • [x] Bug fix (non-breaking change which fixes an issue)

NuclearCookie avatar Feb 11 '20 14:02 NuclearCookie

PR-merge succeeded, but blueocean is reporting it as failed due to JDK not found on windows, so there are no artifacts.

Any clue? https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fgit-client-plugin/detail/PR-505/1/pipeline

NuclearCookie avatar Feb 11 '20 14:02 NuclearCookie

PR-merge succeeded, but blueocean is reporting it as failed due to JDK not found on windows, so there are no artifacts.

Any clue? https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fgit-client-plugin/detail/PR-505/1/pipeline

I'll investigate the ci.jenkins.io failure in more detail. I don't know why it failed. The incorrect recording of success was from the CI server that I run at my house. I've reconfigured it to evaluate pull requests for my personal repository instead of evaluating pull requests for the authoritative repository.

MarkEWaite avatar Feb 11 '20 18:02 MarkEWaite

The failure on ci.jenkins.io was transient. I processed a pull request for another plugin and restarted the build for this plugin. Both did not show any failure installing JDK 8.

MarkEWaite avatar Feb 11 '20 19:02 MarkEWaite

Any news on this? or any (non clean all) workaround until this is merged?

capyvara avatar Mar 06 '20 23:03 capyvara

I'm sorry I was hoping that this would be an easy fix. I currently don't have time to move the tests to the new system and figure out these new changes. I'll try to get back to this PR in a couple of days or weeks.

NuclearCookie avatar Mar 09 '20 07:03 NuclearCookie

Hello @MarkEWaite I finally took the time to update this PR and make the requested changes. I've also ported the tests from JUnit 3 to the new test suite. Since this feature is now behind a flag, how do I go about adding this to the jenkins frontend?

NuclearCookie avatar Aug 27 '20 08:08 NuclearCookie

Hey @MarkEWaite small ping on this one

NuclearCookie avatar Sep 14 '20 13:09 NuclearCookie

@NuclearCookie the merge conflict will need to be resolved. I resolved it in my copy of the branch at https://github.com/MarkEWaite/git-client-plugin/tree/jenkins-55875 if that is any help.

It will be a while before I can address this in more depth. I need to resolve some open issues from the Google Summer of Code work .

MarkEWaite avatar Sep 25 '20 23:09 MarkEWaite

@MarkEWaite I resolved the conflict

NuclearCookie avatar Nov 17 '20 13:11 NuclearCookie

Hello Mark, I would still love to see this PR merged. Do you need anything from my side for that? Thanks

NuclearCookie avatar Jan 25 '21 10:01 NuclearCookie

Hey @MarkEWaite I would love to see this feature merged, it would be a great improvement to our submodule workflow right now. Thanks for taking another look..

NuclearCookie avatar Feb 18 '21 08:02 NuclearCookie

I have been watching this since https://github.com/jenkinsci/git-client-plugin/pull/404, patiently waiting... At this point, jenkins is no longer "The leading open source automation server" in my environments. Several times a day I have to go into the nodes and sometimes the master and clean the submodules manually.

Looking at the current PR, I cannot tell... is there anything I can do to help get this merged? I saw there was a conflict with the PR but that was resolved months ago. If there is still something holding this up, I would be glad to help.

elivaughan avatar Apr 02 '21 13:04 elivaughan

Hello, please merge this PR. My company really needs this to resolve some problems.

michaelakin avatar Apr 02 '21 13:04 michaelakin

Has there been any updates on this? It would be very helpful for me.

Oragoss avatar Apr 02 '21 13:04 Oragoss

please can you merge this PR.

jazreal avatar Apr 02 '21 14:04 jazreal

Hey guys - can we get this PR merged in? Thanks.

Toddman0430 avatar Apr 02 '21 14:04 Toddman0430

@Toddman0430 , @jazreal , @Oragoss , and @michaelakin you can take the plugin development build from https://ci.jenkins.io/job/Plugins/job/git-client-plugin/job/PR-505/lastSuccessfulBuild/artifact/org/jenkins-ci/plugins/git-client/3.7.1-rc2777.9a303a05ed20/git-client-3.7.1-rc2777.9a303a05ed20.hpi and use it on your installation.

If you're confident that this is a solution to problems you're seeing, then you could also consider building your own copy of the git client plugin with this change and the forceSubmoduleUpdate = true instead of the default false. That should address your immediate concern and would allow you to report broader results as you use the change.

MarkEWaite avatar Apr 02 '21 16:04 MarkEWaite

Thanks for the reply @MarkEWaite, in the meantime, is there something I can do to get this change merged?

NuclearCookie avatar Apr 06 '21 07:04 NuclearCookie

Please don't take this wrong... but what is the hold up on getting this merged? Is there a certain number of years that has to pass before this can occur? Is there anything we can do?

elivaughan avatar Jul 01 '21 15:07 elivaughan

@elivaughan, I described in an earlier comment how you could build a version of this change, use it in your local environment, and report your success with the locally built change. That would resolve your issue and increase confidence that there were not unexpected side effects from the change.

Many voices calling for the merge of a change is certainly more persuasive than a single voice calling for a merge, but it is much more persuasive to hear from users that their problem was solved by the pull request.

MarkEWaite avatar Jul 01 '21 18:07 MarkEWaite

Can you merge this PR already? It's been too long. It's very much needed. To avoid unintended consequences we can give force as an optional parameter and default it to false. This way it retains the old behavior unless explicitly changed.

Using plugins of development versions wouldn't be ideal

adityak368 avatar May 31 '22 06:05 adityak368

Another option would be to just have an input field to give arbitrary command line options which would be appended to the command

adityak368 avatar May 31 '22 06:05 adityak368

@adityak368 see the earlier comments. You've added another voice saying, "merge this", but you have not reported that it resolved the issue you're seeing.

Another option that you didn't mention is that you could use the withCredentials support that was added to the git plugin in Google Summer of Code 2021 and that would allow you to call command line git directly with any arbitrary command line options that you choose. See the git plugin documentation for examples

MarkEWaite avatar May 31 '22 10:05 MarkEWaite

Closing this pull request because I can't push changes to its branch. I've save a copy in my repository so that I can open a new pull request to replace it

MarkEWaite avatar May 25 '23 22:05 MarkEWaite