gitlab-plugin icon indicating copy to clipboard operation
gitlab-plugin copied to clipboard

Published 20 hours ago • verified publisher icon jenkinsci

Failed to update Gitlab commit status for project '<project>/<repo>': HTTP 403 Forbidden

Open jackyko1991 opened this issue 6 years ago • 10 comments

Context

  • Gitlab plugin version: 1.5.11
  • Gitlab version: 11.7.0-rc4-ee
  • Jenkins version: 2.157
  • Job type: Pipeline, Freestyle, Matrix, etc.

Logs & Traces

Failed to update Gitlab commit status for project '/': HTTP 403 Forbidden Finished: SUCCESS

Problem description

I have enable post build Gitlab commit status with webhook to official gitlab repos. Seems the UI has changed and gitlab plugin cannot work properly

jackyko1991 avatar Jan 11 '19 16:01 jackyko1991

@jackyko1991 it is not clear from what you have written whether this ever worked for you, or if something changed and now it is broken. A 403 response indicates that the credentials you are using to access the GitLab API do not have the necessary permissions to perform the commit status operation.

omehegan avatar Feb 26 '19 04:02 omehegan

hello i experience the same problem in

  • Gitlab plugin version: 1.5.11
  • Gitlab version: 11.7.5-ce.0
  • Jenkins version: 2.138.2
  • Job type: Freestyle

log

Feb 25, 2019 4:56:57 PM INFO com.dabsquared.gitlabjenkins.webhook.GitLabWebHook getDynamic

WebHook called with url: /project/[FILTERED]

Feb 25, 2019 4:57:12 PM INFO com.dabsquared.gitlabjenkins.webhook.GitLabWebHook getDynamic

WebHook called with url: /project/[FILTERED]

Feb 25, 2019 4:57:12 PM INFO com.dabsquared.gitlabjenkins.trigger.handler.AbstractWebHookTriggerHandler handle

[FILTERED] triggered for merge request.

Feb 25, 2019 4:57:12 PM SEVERE com.dabsquared.gitlabjenkins.trigger.handler.AbstractWebHookTriggerHandler setCommitStatusPendingIfNecessary

Failed to set build state to pending
javax.ws.rs.ClientErrorException: HTTP 403 Forbidden
	at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.handleErrorStatus(ClientInvocation.java:216)
	at org.jboss.resteasy.client.jaxrs.internal.proxy.extractors.DefaultEntityExtractorFactory$3.extractEntity(DefaultEntityExtractorFactory.java:50)
	at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invoke(ClientInvoker.java:104)
	at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientProxy.invoke(ClientProxy.java:64)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.$Proxy96.changeBuildStatus(Unknown Source)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.ResteasyGitLabClient.changeBuildStatus(ResteasyGitLabClient.java:65)
	at com.dabsquared.gitlabjenkins.trigger.handler.AbstractWebHookTriggerHandler.setCommitStatusPendingIfNecessary(AbstractWebHookTriggerHandler.java:74)
	at com.dabsquared.gitlabjenkins.trigger.handler.AbstractWebHookTriggerHandler.handle(AbstractWebHookTriggerHandler.java:53)
	at com.dabsquared.gitlabjenkins.trigger.handler.merge.MergeRequestHookTriggerHandlerImpl.handle(MergeRequestHookTriggerHandlerImpl.java:80)
	at com.dabsquared.gitlabjenkins.trigger.handler.merge.MergeRequestHookTriggerHandlerImpl.handle(MergeRequestHookTriggerHandlerImpl.java:38)
	at com.dabsquared.gitlabjenkins.GitLabPushTrigger.onPost(GitLabPushTrigger.java:456)
	at com.dabsquared.gitlabjenkins.webhook.build.MergeRequestBuildAction$1.performOnPost(MergeRequestBuildAction.java:78)
	at com.dabsquared.gitlabjenkins.webhook.build.BuildWebHookAction$TriggerNotifier.run(BuildWebHookAction.java:54)
	at hudson.security.ACL.impersonate(ACL.java:273)
	at com.dabsquared.gitlabjenkins.webhook.build.MergeRequestBuildAction.execute(MergeRequestBuildAction.java:75)
	at com.dabsquared.gitlabjenkins.webhook.build.BuildWebHookAction.execute(BuildWebHookAction.java:31)
	at com.dabsquared.gitlabjenkins.webhook.GitLabWebHook.getDynamic(GitLabWebHook.java:44)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:343)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:184)
	at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:372)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:734)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:864)
	at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:734)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:864)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:668)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
	at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:243)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:59)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at com.dabsquared.gitlabjenkins.webhook.GitLabWebHook$GitlabWebHookCrumbExclusion.process(GitLabWebHook.java:53)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:73)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:531)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
	at java.lang.Thread.run(Thread.java:748)

Feb 25, 2019 4:57:20 PM INFO com.dabsquared.gitlabjenkins.util.CommitStatusUpdater retrieveGitlabProjectIds

Retrieving gitlab project ids

Feb 25, 2019 4:57:21 PM INFO com.dabsquared.gitlabjenkins.util.CommitStatusUpdater updateCommitStatus

Updating build '14' to 'running'

Feb 25, 2019 4:57:21 PM SEVERE com.dabsquared.gitlabjenkins.util.CommitStatusUpdater updateCommitStatus

Failed to update Gitlab commit status for project '14'
javax.ws.rs.ClientErrorException: HTTP 403 Forbidden
	at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.handleErrorStatus(ClientInvocation.java:216)
	at org.jboss.resteasy.client.jaxrs.internal.proxy.extractors.DefaultEntityExtractorFactory$3.extractEntity(DefaultEntityExtractorFactory.java:50)
	at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invoke(ClientInvoker.java:104)
	at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientProxy.invoke(ClientProxy.java:64)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.$Proxy96.changeBuildStatus(Unknown Source)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.ResteasyGitLabClient.changeBuildStatus(ResteasyGitLabClient.java:60)
	at com.dabsquared.gitlabjenkins.util.CommitStatusUpdater.updateCommitStatus(CommitStatusUpdater.java:85)
	at com.dabsquared.gitlabjenkins.util.CommitStatusUpdater.updateCommitStatus(CommitStatusUpdater.java:97)
	at com.dabsquared.gitlabjenkins.publisher.GitLabCommitStatusPublisher.prebuild(GitLabCommitStatusPublisher.java:45)
	at hudson.model.AbstractBuild$AbstractBuildExecution.preBuild(AbstractBuild.java:798)
	at hudson.model.AbstractBuild$AbstractBuildExecution.preBuild(AbstractBuild.java:793)
	at hudson.model.Build$BuildExecution.doRun(Build.java:145)
	at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
	at hudson.model.Run.execute(Run.java:1819)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)

Feb 25, 2019 5:04:48 PM INFO hudson.model.Run execute

[FILTERED] #69 main build action completed: SUCCESS

what i noticed though is that the jenkins job is trying to push the status to the source project from the MR which i was not expecting. project 14 is the source project in the MR. my gitlab user has no access to that project, which i think makes sense. i would like to update only the MR for which i configured the post-build action (at least that's what i thought the post-build action would do).

  1. What's the expected behavior of this post-build action?
  2. Is there configuration options for it?

ludiofines avatar Feb 27 '19 15:02 ludiofines

Hi @omehegan any news on the issue?

ludiofines avatar Mar 11 '19 14:03 ludiofines

Not sure if this is relevant for anyone else; I was seeing this behaviour and it turned out to be attempting to go through our proxy server when setting the commit status. I added a bypass (see https://wiki.jenkins.io/display/JENKINS/JenkinsBehindProxy) and it is now working.

flagmanchris avatar Nov 27 '19 15:11 flagmanchris

Recently had jenkins updates, plugin updates, and gitlab updates go down, and now I'm seeing this on a prior working configuration. Last working state was a month ago.

Access levels haven't changed at all for the Jenkins user, and I can validate that authentication hasn't changed, and this now failing gitlab projects / jenkins builds that were working prior.

Running:

  • Jenkins 2.204.5 LTS (slightly tweaked Docker image for running Jenkins as a non-root user)
  • GitLab Plugin for Jenkins v1.5.13
  • GitLab CE v12.8.6 (5fc76a64537)

Stack trace:

Mar 16, 2020 12:54:52 PM SEVERE com.dabsquared.gitlabjenkins.util.CommitStatusUpdater updateCommitStatus

Failed to update Gitlab commit status for project 'katana/TreeResolver'
javax.ws.rs.ClientErrorException: HTTP 403 Forbidden
	at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.handleErrorStatus(ClientInvocation.java:216)
	at org.jboss.resteasy.client.jaxrs.internal.proxy.extractors.DefaultEntityExtractorFactory$3.extractEntity(DefaultEntityExtractorFactory.java:50)
	at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invoke(ClientInvoker.java:104)
	at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientProxy.invoke(ClientProxy.java:64)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.$Proxy121.changeBuildStatus(Unknown Source)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.ResteasyGitLabClient.changeBuildStatus(ResteasyGitLabClient.java:60)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$7.execute(AutodetectingGitLabClient.java:112)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$7.execute(AutodetectingGitLabClient.java:109)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$GitLabOperation.execute(AutodetectingGitLabClient.java:335)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$GitLabOperation.access$100(AutodetectingGitLabClient.java:332)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient.execute(AutodetectingGitLabClient.java:328)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient.changeBuildStatus(AutodetectingGitLabClient.java:108)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$7.execute(AutodetectingGitLabClient.java:112)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$7.execute(AutodetectingGitLabClient.java:109)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$GitLabOperation.execute(AutodetectingGitLabClient.java:335)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$GitLabOperation.access$100(AutodetectingGitLabClient.java:332)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient.execute(AutodetectingGitLabClient.java:328)
	at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient.changeBuildStatus(AutodetectingGitLabClient.java:108)
	at com.dabsquared.gitlabjenkins.util.CommitStatusUpdater.updateCommitStatus(CommitStatusUpdater.java:85)
	at com.dabsquared.gitlabjenkins.util.CommitStatusUpdater.updateCommitStatus(CommitStatusUpdater.java:97)
	at com.dabsquared.gitlabjenkins.workflow.UpdateGitLabCommitStatusStep$UpdateGitLabCommitStatusStepExecution.run(UpdateGitLabCommitStatusStep.java:79)
	at com.dabsquared.gitlabjenkins.workflow.UpdateGitLabCommitStatusStep$UpdateGitLabCommitStatusStepExecution.run(UpdateGitLabCommitStatusStep.java:63)
	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousStepExecution.start(AbstractSynchronousStepExecution.java:42)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:286)
	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:179)
	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:122)
	at sun.reflect.GeneratedMethodAccessor338.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
	at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
	at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
	at org.codehaus.groovy.runtime.callsite.PogoMetaClassSite.call(PogoMetaClassSite.java:42)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:163)
	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:23)
	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:157)
	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:135)
	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:86)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:113)
	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:83)
	at sun.reflect.GeneratedMethodAccessor159.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.dispatch(CollectionLiteralBlock.java:55)
	at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.item(CollectionLiteralBlock.java:45)
	at sun.reflect.GeneratedMethodAccessor162.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
	at com.cloudbees.groovy.cps.impl.ConstantBlock.eval(ConstantBlock.java:21)
	at com.cloudbees.groovy.cps.Next.step(Next.java:83)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:174)
	at com.cloudbees.groovy.cps.Continuable$1.call(Continuable.java:163)
	at org.codehaus.groovy.runtime.GroovyCategorySupport$ThreadCategoryInfo.use(GroovyCategorySupport.java:129)
	at org.codehaus.groovy.runtime.GroovyCategorySupport.use(GroovyCategorySupport.java:268)
	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:163)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:51)
	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:185)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:400)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:96)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:312)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:276)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I have a pending update for GitLab that needs to be applied, so I'll give that a whirl and see if it's a GitLab regression, but going by the changelog, I doubt it.

GitLab update did not help. Tried rolling back to prior Gitlab plugin version (1.5.12), no effect.

Reissued token to no avail, as well.

GitLab logs:

==> /var/log/gitlab/nginx/gitlab_access.log <==
[REDACTED] - - [18/Mar/2020:16:18:39 +0000] "GET /api/v4/projects/katana%2FTreeResolver/repository/commits/f1c35fef86bb22a8d067d94636f649e0cc955332 HTTP/1.1" 200 878 "" "-"
[REDACTED] - - [18/Mar/2020:16:18:39 +0000] "POST /api/v4/projects/katana%2FTreeResolver/statuses/f1c35fef86bb22a8d067d94636f649e0cc955332 HTTP/1.1" 403 27 "" "-"
[REDACTED] - - [18/Mar/2020:16:18:39 +0000] "GET /api/v4/projects/katana%2FTreeResolver/repository/commits/f1c35fef86bb22a8d067d94636f649e0cc955332 HTTP/1.1" 200 878 "" "-"
[REDACTED] - - [18/Mar/2020:16:18:39 +0000] "POST /api/v4/projects/katana%2FTreeResolver/statuses/f1c35fef86bb22a8d067d94636f649e0cc955332 HTTP/1.1" 403 27 "" "-"

==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/api/v4/projects/katana%2FTreeResolver/repository/commits/f1c35fef86bb22a8d067d94636f649e0cc955332" for [REDACTED] at 2020-03-18 16:18:39 +0000
Started POST "/api/v4/projects/katana%2FTreeResolver/statuses/f1c35fef86bb22a8d067d94636f649e0cc955332" for [REDACTED] at 2020-03-18 16:18:39 +0000
Started GET "/api/v4/projects/katana%2FTreeResolver/repository/commits/f1c35fef86bb22a8d067d94636f649e0cc955332" for [REDACTED] at 2020-03-18 16:18:39 +0000
Started POST "/api/v4/projects/katana%2FTreeResolver/statuses/f1c35fef86bb22a8d067d94636f649e0cc955332" for [REDACTED] at 2020-03-18 16:18:39 +0000

==> /var/log/gitlab/gitlab-rails/api_json.log <==
{"time":"2020-03-18T16:18:39.347Z","severity":"INFO","duration":47.85,"db":5.62,"view":42.230000000000004,"status":200,"method":"GET","path":"/api/v4/projects/katana%2FTreeResolver/repository/commits/f1c35fef86bb22a8d067d94636f649e0cc955332","params":[],"host":"gitlab.[REDACTED]","remote_ip":"[REDACTED], [REDACTED], 127.0.0.1","ua":null,"route":"/api/:version/projects/:id/repository/commits/:sha","user_id":4,"username":"jenkins","queue_duration":6.58,"gitaly_calls":1,"gitaly_duration":7.04,"rugged_calls":1,"rugged_duration_ms":4.4,"correlation_id":"T6FIgeHJH81"}
{"time":"2020-03-18T16:18:39.381Z","severity":"INFO","duration":24.85,"db":4.88,"view":19.970000000000002,"status":403,"method":"POST","path":"/api/v4/projects/katana%2FTreeResolver/statuses/f1c35fef86bb22a8d067d94636f649e0cc955332","params":[{"key":"state","value":"pending"},{"key":"context","value":"jenkins"},{"key":"target_url","value":"https://jenkins.[REDACTED]/job/TreeResolver/job/master/31/display/redirect"},{"key":"description","value":"[FILTERED]"}],"host":"gitlab.[REDACTED]","remote_ip":"[REDACTED], [REDACTED], 127.0.0.1","ua":null,"route":"/api/:version/projects/:id/statuses/:sha","user_id":4,"username":"jenkins","queue_duration":3.56,"rugged_calls":1,"rugged_duration_ms":3.52,"correlation_id":"V4mr3cdrUI9"}
{"time":"2020-03-18T16:18:39.571Z","severity":"INFO","duration":51.41,"db":8.41,"view":43.0,"status":200,"method":"GET","path":"/api/v4/projects/katana%2FTreeResolver/repository/commits/f1c35fef86bb22a8d067d94636f649e0cc955332","params":[],"host":"gitlab.[REDACTED]","remote_ip":"[REDACTED], [REDACTED], 127.0.0.1","ua":null,"route":"/api/:version/projects/:id/repository/commits/:sha","user_id":4,"username":"jenkins","queue_duration":8.07,"gitaly_calls":1,"gitaly_duration":7.66,"rugged_calls":1,"rugged_duration_ms":1.85,"correlation_id":"EoMi9vlS0b8"}
{"time":"2020-03-18T16:18:39.608Z","severity":"INFO","duration":28.14,"db":5.16,"view":22.98,"status":403,"method":"POST","path":"/api/v4/projects/katana%2FTreeResolver/statuses/f1c35fef86bb22a8d067d94636f649e0cc955332","params":[{"key":"state","value":"running"},{"key":"context","value":"jenkins"},{"key":"target_url","value":"https://jenkins.[REDACTED]/job/TreeResolver/job/master/31/display/redirect"},{"key":"description","value":"[FILTERED]"}],"host":"gitlab.[REDACTED]","remote_ip":"[REDACTED], [REDACTED], 127.0.0.1","ua":null,"route":"/api/:version/projects/:id/statuses/:sha","user_id":4,"username":"jenkins","queue_duration":3.88,"rugged_calls":1,"rugged_duration_ms":3.94,"correlation_id":"WIIHfLOqEka"}

==> /var/log/gitlab/gitaly/current <==
{"correlation_id":"T6FIgeHJH81","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.method":"CommitStats","grpc.request.deadline":"2020-03-18T16:19:09Z","grpc.request.fullMethod":"/gitaly.CommitService/CommitStats","grpc.request.glProjectPath":"katana/TreeResolver","grpc.request.glRepository":"project-19","grpc.request.repoPath":"katana/TreeResolver.git","grpc.request.repoStorage":"default","grpc.request.topLevelGroup":"katana","grpc.service":"gitaly.CommitService","grpc.start_time":"2020-03-18T16:18:39Z","grpc.time_ms":5.839,"level":"info","msg":"finished unary call with code OK","peer.address":"@","pid":938,"span.kind":"server","system":"grpc","time":"2020-03-18T16:18:39Z"}
{"correlation_id":"EoMi9vlS0b8","grpc.code":"OK","grpc.meta.auth_version":"v2","grpc.meta.client_name":"gitlab-web","grpc.meta.deadline_type":"regular","grpc.method":"CommitStats","grpc.request.deadline":"2020-03-18T16:19:09Z","grpc.request.fullMethod":"/gitaly.CommitService/CommitStats","grpc.request.glProjectPath":"katana/TreeResolver","grpc.request.glRepository":"project-19","grpc.request.repoPath":"katana/TreeResolver.git","grpc.request.repoStorage":"default","grpc.request.topLevelGroup":"katana","grpc.service":"gitaly.CommitService","grpc.start_time":"2020-03-18T16:18:39Z","grpc.time_ms":6.076,"level":"info","msg":"finished unary call with code OK","peer.address":"@","pid":938,"span.kind":"server","system":"grpc","time":"2020-03-18T16:18:39Z"}

==> /var/log/gitlab/gitlab-workhorse/current <==
{"correlation_id":"T6FIgeHJH81","duration_ms":55,"host":"gitlab.[REDACTED]","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":200,"system":"http","time":"2020-03-18T16:18:39Z","uri":"/api/v4/projects/katana%2FTreeResolver/repository/commits/f1c35fef86bb22a8d067d94636f649e0cc955332","user_agent":"","written_bytes":878}
{"correlation_id":"V4mr3cdrUI9","duration_ms":29,"host":"gitlab.[REDACTED]","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":403,"system":"http","time":"2020-03-18T16:18:39Z","uri":"/api/v4/projects/katana%2FTreeResolver/statuses/f1c35fef86bb22a8d067d94636f649e0cc955332","user_agent":"","written_bytes":27}
{"correlation_id":"EoMi9vlS0b8","duration_ms":60,"host":"gitlab.[REDACTED]","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":200,"system":"http","time":"2020-03-18T16:18:39Z","uri":"/api/v4/projects/katana%2FTreeResolver/repository/commits/f1c35fef86bb22a8d067d94636f649e0cc955332","user_agent":"","written_bytes":878}
{"correlation_id":"WIIHfLOqEka","duration_ms":32,"host":"gitlab.[REDACTED]","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":403,"system":"http","time":"2020-03-18T16:18:39Z","uri":"/api/v4/projects/katana%2FTreeResolver/statuses/f1c35fef86bb22a8d067d94636f649e0cc955332","user_agent":"","written_bytes":27}

katanacrimson avatar Mar 18 '20 15:03 katanacrimson

Potentially a GitLab regression, at least for me: https://gitlab.com/gitlab-org/gitlab/-/issues/210389

katanacrimson avatar Mar 18 '20 16:03 katanacrimson

I solved this issue by changing permissions to Maintainer level for jenkins user in gitlab

pysiekytel avatar Jul 06 '20 12:07 pysiekytel

I had this issue when dealing with a merge request created from a fork. My error message was Failed to update Gitlab commit status for project '1234': HTTP 403 Forbidden

The project ID referenced in the error is the source project, and not the target project, of the merge request was created against. Granting Developer rights on the fork to the user used by Jenkins solves the problem.

While the MR object belongs to the target project, the pipeline object it contains is has a web_url reference to the source project. I suppose that accessing and/or modifying that pipeline object is why the Jenkins user must have access to the source project.

Roming22 avatar Mar 05 '21 13:03 Roming22

I solved this issue by changing permissions to Maintainer level for jenkins user in gitlab

I always forget this setting! I think this setting could be emphasized in documentation

ukuko avatar Dec 15 '22 17:12 ukuko

I solved this issue by changing permissions to Maintainer level for jenkins user in gitlab

Did works to me!!! Tks. In my case, user Jenkins has other name in my environment... check the Jenkins login before to add in your git project.

thiagoinhesta avatar Mar 31 '23 20:03 thiagoinhesta