JGit+ssh issues

[imavroukakis]

We're pushing releases from our CI server into github via SSH authentication over corkscrew to bypass certain network restrictions. Even though I've pointed axion to our key, it doesn't seem to be able to do it.

Pushing all to remote: origin
:release FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':release'.
> org.eclipse.jgit.api.errors.TransportException: [email protected]:myorg/my-project.git: Connection timed out

Is there any way around this?

[adamdubiel]

Okay, can you show me how did you point to the key file?

[imavroukakis]

Using release.customKeyFile and pointing it to the SSH RSA key of the account that's permissioned for GitHub access (namely, the user that runs the CI)

[imavroukakis]

To be more precise clean test release -Prelease.disableChecks -Prelease.pushTagsOnly -Prelease.customKeyFile="/home/jenkins/.ssh/id_rsa"

[adamdubiel]

Oh, wait, it's actually connection timeout? Don't know about your network setup, but are you sure the 22 is open? We could use -Djavax.net.debug=all to get some more info from network layer itself.

[imavroukakis]

No and that's the point I'm making. 22 isn't open, hence we use corkscrew to tunnel ssh traffic through our web proxy. Normally, this is something that's transparent to git as it's done by the ssh configuration, but JGit is a completely different beast...

[adamdubiel]

Ah, okay - i never used corkscrew so i'm not familiar with it. JGIt is actually Git implementation in Java, it does not use git (at least i'm not aware of it), so it may behave totally different in such cases. The only option would be to replace jgit API with native git calls, but its a great deal of work.

[imavroukakis]

It's actually quite common to use corkscrew or some other proxy, to route [email protected] requests to ssh.github.com:443 via a proxy..think this can be done as things stand?

[adamdubiel]

I need to do some research.

[imavroukakis]

Thanks again. GitHub themselves have setup ssh.github.com:443 just because some corporate firewalls can be overly restrictive..

[adamdubiel]

Never had issue with this, so it's all a new problem, but thanks for pointing it out :)

[imavroukakis]

Some interesting reading https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin#GitPlugin-WhyNotJGit So in essence, you can use https://wiki.jenkins-ci.org/display/JENKINS/Git+Client+Plugin and have it as a configurable option whether the local git instance will be used or JGit

[adamdubiel]

I'm not that familiar with Jenkins, but from what i see i would need to introduce changes in the code to use this client-api instead of JGit when running on Jenkins to make most of Jenkins-related problems go away?

[imavroukakis]

In a nutshell yes. But the main issue seems to be that JGit simply doesn't provide parity with the git CLI. What git-client provides is essentially a bridge between the two, until JGit finally catches up.

[imavroukakis]

oh and a very Happy New Year :smiley_cat:

[adamdubiel]

Thanks :) Still 3 hours to go in my timezone :D

Okay, i will look into it, i know JGit has a lot of issues, but in general it works well.

[jgoldhammer]

Hi Adam,

thanks for providing axion-release-plugin. Unfortunately, this is currently a blocker for us. What´s the state of changing to git-client?

Thanks, Jens

[adamdubiel]

I haven't estimated the cost of this yet. Frankly speaking it would be ideal if this came as a pull-request, since i have no experience with Jenkins.

[imavroukakis]

We'll see what we can do :)

[Pesegato]

I did

gradlew markNextVersion -Prelease.version=0.4.0 -Prelease.customKeyFile="C:\Users\me\.ssh\github_rsa"

And got

Execution failed for task ':markNextVersion'.
> Cannot cast object 'org.eclipse.jgit.transport.TransportHttp@ad3f70a' with cla
ss 'org.eclipse.jgit.transport.TransportHttp' to class 'org.eclipse.jgit.transpo
rt.SshTransport'
Caused by: org.codehaus.groovy.runtime.typehandling.GroovyCastException: Cannot
cast object 'org.eclipse.jgit.transport.TransportHttp@ad3f70a' with class 'org.e
clipse.jgit.transport.TransportHttp' to class 'org.eclipse.jgit.transport.SshTra
nsport'
        at pl.allegro.tech.build.axion.release.infrastructure.git.TransportConfi
gFactory$1.configure(TransportConfigFactory.groovy:23)
        at org.eclipse.jgit.api.TransportCommand.configure(TransportCommand.java
:138)
        at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:151)
        at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:80)

[VimalRaveendran]

• @adamdubiel I had the same issue (Cannot cast object 'org.eclipse.jgit.transport.TransportHttp@ad3f70a' with cla ss 'org.eclipse.jgit.transport.TransportHttp' to class 'org.eclipse.jgit.transpo rt.SshTransport') while using a ssh key. Is there any work around for this yet? Also, like the comments on Issue-86, I tried all possible way to authenticate my enterprise git repo, but nothing works. I keep getting "cannot open git-receive-pack:"
• @Pesegato Did you find your way around this issue? Any help or suggestion would be great.

[adamdubiel]

No, not really. I was not able to find any way to reproduce any of those issues. Just for reference, are you using the newest version of plugin (1.8.1)? I know i updated JGit to newest version recently.

Also currently i'm working on tests that use Apache Mina SSHD to replicate some issues i encountered, so once they are there, maybe you could use them to reproduce the issue you are observing by mirroring your remote SSH servicer configuration in in-memory SSHD?

[VimalRaveendran]

Thanks for the response @adamdubiel. Yes, I'm using the latest version. But strangely, I dont even get into interactive mode if I dont specify cred's. I guess it picks up the cred's from osx keychain, but that doesn't work either (resulting in cannot open git-receive-pack).

[mario16]

Hi, I recently had the same error. If you try to authenticate via ssh you need to use the URI repository for ssh. For example: ssh://[email protected]:githubtraining/hellogitworld.git Try it and tell me.

[VimalRaveendran]

@mario16 Thanks for the response. Could you point me to a working repo in github where you have done the url change. I'm not sure how to set the url as i dont know which property to configure.

[mario16]

I assume you are discussing to authenticate a git repo using SSH. You need to follow this example -> http://www.codeaffine.com/2014/12/09/jgit-authentication/ And when you set: cloneCommand.setURI( "ssh://[email protected]/repo.git" ); Indicate the url with ssh protocol, the repo in github I named is active (ssh://[email protected]:githubtraining/hellogitworld.git).

[georgThesis]

@mario16 where did you put this in your build.gradle? Could you pass an example? Is it meant to go inside axion-release scmVersion block?

I face the same problem and believe it is related to Gitlab CI cloning the project via HTTPS and me specifying an ssh key instead of the gitlab access token

[toniedzwiedz]

Not sure this is related but I am really struggling with getting Axion to work with my SSH key. I have a passphrase set up and I'm running Cmder with ssh-agent active on Windows 10. SSH, Git, scp and friends work just fine. I'm prompted for a passphrase as the terminal starts, then it's smooth sailing.

But it is not so with Axion. I'm trying to perform a release and I keep getting the following stack trace:

 FAILURE: Build failed with an exception.

 * What went wrong:
 Execution failed for task ':release'.
 > org.eclipse.jgit.api.errors.TransportException: ssh://[email protected]/~tomasz.niedzwiedz/jabberwocky.git: USERAUTH fail

 * Try:
  Run with --scan to get full insights.

 * Exception is:
 org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':release'.
   at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:110)
   at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77)
   at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)   at org.gradle.api.internal.tasks.execution.SkipCachedTaskExecuter.execute(SkipCachedTaskExecuter.java:105)
   at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
   at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
   at org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:79)
   at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
   at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
   at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
   at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
   at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
   at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
   at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
   at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
   at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
   at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
   at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
   at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
   at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
   at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
   at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
   at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
   at org.gradle.execution.taskgraph.LocalTaskInfoExecutor.execute(LocalTaskInfoExecutor.java:42)
   at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:277)
   at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareWorkItemExecutor.execute(DefaultTaskExecutionGraph.java:262)
   at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:135)
   at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130)
   at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.execute(DefaultTaskPlanExecutor.java:200)
   at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.executeWithWork(DefaultTaskPlanExecutor.java:191)
   at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$ExecutorWorker.run(DefaultTaskPlanExecutor.java:130)
   at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
   at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
   at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
 Caused by: pl.allegro.tech.build.axion.release.domain.scm.ScmException: org.eclipse.jgit.api.errors.TransportException: ssh://[email protected]/~tomasz.niedzwiedz/jabberwocky.git: USERAUTH fail
   at pl.allegro.tech.build.axion.release.infrastructure.git.GitRepository.callPush(GitRepository.groovy:122)
   at pl.allegro.tech.build.axion.release.infrastructure.git.GitRepository.push(GitRepository.groovy:108)
   at pl.allegro.tech.build.axion.release.infrastructure.git.GitRepository.push(GitRepository.groovy:99)
   at pl.allegro.tech.build.axion.release.domain.scm.ScmRepository$push$5.call(Unknown Source)
   at pl.allegro.tech.build.axion.release.domain.scm.ScmService.push(ScmService.groovy:43)
   at pl.allegro.tech.build.axion.release.domain.scm.ScmService$push$1.call(Unknown Source)
   at pl.allegro.tech.build.axion.release.domain.Releaser.pushRelease(Releaser.groovy:58)
   at pl.allegro.tech.build.axion.release.domain.Releaser.releaseAndPush(Releaser.groovy:49)
   at pl.allegro.tech.build.axion.release.domain.Releaser$releaseAndPush.call(Unknown Source)
   at pl.allegro.tech.build.axion.release.ReleaseTask.release(ReleaseTask.groovy:22)
   at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
   at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:46)
   at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:39)
   at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:26)
   at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:801)
   at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:768)
   at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:131)
   at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:300)
   at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:292)
   at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:174)
   at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
   at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
   at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120)
   at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99)
   ... 33 more
 Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://[email protected]/~tomasz.niedzwiedz/jabberwocky.git: USERAUTH fail
   at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:180)
   at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:85)
   at java_util_concurrent_Callable$call.call(Unknown Source)
   at pl.allegro.tech.build.axion.release.infrastructure.git.GitRepository.callPush(GitRepository.groovy:120)
   ... 56 more
 Caused by: org.eclipse.jgit.errors.TransportException: ssh://[email protected]/~tomasz.niedzwiedz/jabberwocky.git: USERAUTH fail
   at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:182)
   at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:140)
   at org.eclipse.jgit.transport.TransportGitSsh$SshPushConnection.<init>(TransportGitSsh.java:338)
   at org.eclipse.jgit.transport.TransportGitSsh.openPush(TransportGitSsh.java:176)
   at org.eclipse.jgit.transport.PushProcess.execute(PushProcess.java:160)
   at org.eclipse.jgit.transport.Transport.push(Transport.java:1344)
   at org.eclipse.jgit.api.PushCommand.call(PushCommand.java:169)
   ... 59 more
 Caused by: com.jcraft.jsch.JSchException: USERAUTH fail
   at com.jcraft.jsch.UserAuthPublicKey.start(UserAuthPublicKey.java:119)
   at com.jcraft.jsch.Session.connect(Session.java:470)
   at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:136)
   ... 65 more

I can push to the very same repo with git itself within the same terminal session without any issues or prompts.

Using id 'pl.allegro.tech.build.axion-release' version "1.9.2". Also tried axion-release-1.10.1 with the same effect.

Interestingly, as far as I can tell, SshAgentIdentityRepositoryFactory appers to try to use SSH agent but I don't see any of the errors defined there.