jenkinsci
Intermittent 400 Bad Request errors
Jenkins and plugins versions report
Environment
Jenkins: 2.346.3
OS: Linux - 5.15.0-1026-aws
Java: 11.0.16.1 - Eclipse Adoptium (OpenJDK 64-Bit Server VM)
---
ace-editor:1.1
allure-jenkins-plugin:2.30.2
analysis-model-api:10.16.1
anchore-container-scanner:1.0.24
ansicolor:1.0.2
ant:475.vf34069fef73c
antisamy-markup-formatter:2.7
apache-httpcomponents-client-4-api:4.5.13-138.v4e7d9a_7b_a_e61
artifact-manager-s3:670.v0558a_cb_c82c2
artifactory:3.17.4
audit-trail:3.11
authentication-tokens:1.4
aws-credentials:191.vcb_f183ce58b_9
aws-global-configuration:100.v4619b_45d2dfc
aws-java-sdk-ec2:1.12.287-357.vf82d85a_6eefd
aws-java-sdk-minimal:1.12.287-357.vf82d85a_6eefd
blackduck-detect:7.0.0
blueocean:1.25.7
blueocean-autofavorite:1.2.5
blueocean-bitbucket-pipeline:1.25.7
blueocean-commons:1.25.7
blueocean-config:1.25.7
blueocean-core-js:1.25.7
blueocean-dashboard:1.25.7
blueocean-display-url:2.4.1
blueocean-events:1.25.7
blueocean-git-pipeline:1.25.7
blueocean-github-pipeline:1.25.7
blueocean-i18n:1.25.7
blueocean-jira:1.25.7
blueocean-jwt:1.25.7
blueocean-personalization:1.25.7
blueocean-pipeline-api-impl:1.25.7
blueocean-pipeline-editor:1.25.7
blueocean-pipeline-scm-api:1.25.7
blueocean-rest:1.25.7
blueocean-rest-impl:1.25.7
blueocean-web:1.25.7
bootstrap5-api:5.2.0-3
bouncycastle-api:2.26
branch-api:2.1046.v0ca_37783ecc5
build-blocker-plugin:1.7.8
build-monitor-plugin:1.13+build.202205140447
build-user-vars-plugin:1.8
caffeine-api:2.9.3-65.v6a_47d0f4d1fe
checks-api:1.7.5
cloudbees-bitbucket-branch-source:791.vb_eea_a_476405b
cloudbees-folder:6.740.ve4f4ffa_dea_54
clover:4.13.0
command-launcher:1.2
commons-lang3-api:3.12.0-36.vd97de6465d5b_
conditional-buildstep:1.4.2
config-file-provider:3.11.1
configuration-as-code:1512.vb_79d418d5fc8
configuration-as-code-groovy:1.1
copyartifact:1.47
cors-filter:1.1
credentials:1143.vb_e8b_b_ceee347
credentials-binding:523.vd859a_4b_122e6
cucumber-reports:5.7.3
dark-theme:245.vb_a_2b_b_010ea_96
dashboard-view:2.447.vda_124dd35f11
data-tables-api:1.12.1-3
description-setter:1.10
display-url-api:2.3.6
docker-commons:1.21
docker-workflow:521.v1a_a_dd2073b_2e
dtkit-api:3.0.1
durable-task:500.v8927d9fd99d8
echarts-api:5.3.3-1
email-ext:2.91
embeddable-build-status:255.va_d2370ee8fde
envinject:2.875.v9b_9e962da_a_ec
envinject-api:1.199.v3ce31253ed13
extended-read-permission:3.2
extensible-choice-parameter:1.8.0
external-monitor-job:192.ve979ca_8b_3ccd
favorite:2.4.1
file-parameters:205.vf6ce13b_e5dee
font-awesome-api:6.1.2-1
forensics-api:1.15.1
gerrit-trigger:2.37.1215-NOKIA
git:4.11.5
git-client:3.11.2
git-server:1.11
github:1.34.5
github-api:1.303-400.v35c2d8258028
github-branch-source:1694.vd46793a_c4a_57
gitlab-api:5.0.1-78.v47a_45b_9f78b_7
gitlab-branch-source:642.v9ed86b_b_54384
gitlab-plugin:1.5.35
gradle:1.39.4
groovy:442.v817e6d937d6c
handlebars:3.0.8
handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953
htmlpublisher:1.30
http_request:1.16
ionicons-api:24.va_745a_b_a_97fc3
ivy:2.2
jackson2-api:2.13.3-285.vc03c0256d517
jacoco:3.3.2
jakarta-activation-api:2.0.1-1
jakarta-mail-api:2.0.1-1
javadoc:226.v71211feb_e7e9
javax-activation-api:1.2.0-4
javax-mail-api:1.6.2-5
jaxb:2.3.6-1
jdk-tool:55.v1b_32b_6ca_f9ca
jenkins-design-language:1.25.7
jersey2-api:2.37-1
jira:3.8
jjwt-api:0.11.5-77.v646c772fddb_0
job-dsl:1.81
job-restrictions:0.8
jobConfigHistory:1171.v04b_66d78555e
jquery:1.12.4-1
jquery-detached:1.2.1
jquery3-api:3.6.0-4
jsch:0.1.55.61.va_e9ee26616e7
junit:1119.1121.vc43d0fc45561
kubernetes:1.31.3
kubernetes-client-api:5.12.2-193.v26a_6078f65a_9
kubernetes-credentials:0.9.0
ldap:2.12
leastload:3.0.0
locale:180.v207501dff9b_a_
lockable-resources:2.16
mailer:435.v79ef3972b_5c7
mapdb-api:1.0.9-28.vf251ce40855d
mask-passwords:3.3
matrix-project:772.v494f19991984
maven-plugin:3.19
mercurial:1251.va_b_121f184902
metrics:4.2.10-389.v93143621b_050
mina-sshd-api-common:2.8.0-36.v8e25ce90d4b_1
mina-sshd-api-core:2.8.0-36.v8e25ce90d4b_1
momentjs:1.1.1
multibranch-build-strategy-extension:1.0.10
next-build-number:1.8
okhttp-api:4.9.3-108.v0feda04578cf
overops-query:3.0.3
pam-auth:1.10
parameter-separator:1.3
parameterized-scheduler:1.1
parameterized-trigger:2.45
performance:3.20
pipeline-build-step:2.18
pipeline-graph-analysis:195.v5812d95a_a_2f9
pipeline-groovy-lib:612.v84da_9c54906d
pipeline-input-step:449.v77f0e8b_845c4
pipeline-milestone-step:101.vd572fef9d926
pipeline-model-api:2.2114.v2654ca_721309
pipeline-model-definition:2.2114.v2654ca_721309
pipeline-model-extensions:2.2114.v2654ca_721309
pipeline-rest-api:2.24
pipeline-stage-step:296.v5f6908f017a_5
pipeline-stage-tags-metadata:2.2114.v2654ca_721309
pipeline-stage-view:2.24
pipeline-utility-steps:2.13.0
plain-credentials:139.ved2b_9cf7587b
plugin-util-api:2.17.0
popper2-api:2.11.6-1
postbuildscript:3.1.0-375.v3db_cd92485e1
prism-api:1.28.0-2
prometheus:2.0.11
pubsub-light:1.16
rebuild:1.34
resource-disposer:0.20
robot:3.2.0
role-strategy:561.v9846c7351a_41
run-condition:1.5
saml:2.333.vc81e525974a_c
scm-api:621.vda_a_b_055e58f7
script-security:1175.v4b_d517d6db_f0
sidebar-link:2.2.0
simple-theme-plugin:103.va_161d09c38c7
snakeyaml-api:1.31-84.ve43da_fb_49d0b
sonar:2.14
sse-gateway:1.25
ssh-agent:295.v9ca_a_1c7cc3a_a_
ssh-credentials:305.v8f4381501156
sshd:3.242.va_db_9da_b_26a_c3
structs:324.va_f5d6774f3a_d
subversion:2.16.0
support-core:1206.v14049fa_b_d860
synopsys-coverity:3.0.1
test-results-analyzer:0.3.5
theme-manager:1.5
throttle-concurrents:2.9
timestamper:1.19
token-macro:308.v4f2b_ed62b_b_16
trilead-api:1.67.vc3938a_35172f
uno-choice:2.6.4
urltrigger:1.02
validating-string-parameter:2.8
variant:59.vf075fe829ccb
view-job-filters:2.3
warnings-ng:9.19.1
workflow-aggregator:590.v6a_d052e5a_a_b_5
workflow-api:1192.v2d0deb_19d212
workflow-basic-steps:994.vd57e3ca_46d24
workflow-cps:2759.v87459c4eea_ca_
workflow-durable-task-step:1199.v02b_9244f8064
workflow-job:1207.ve6191ff089f8
workflow-multibranch:716.vc692a_e52371b_
workflow-scm-step:400.v6b_89a_1317c9a_
workflow-step-api:639.v6eca_cd8c04a_a_
workflow-support:838.va_3a_087b_4055b
ws-cleanup:0.43
xray-connector:2.6.1
xtrigger-api:0.4
xunit:3.1.0
What Operating System are you using (both controller, and any agents involved in the problem)?
Ubuntu 22.04.1 LTS for both
Reproduction steps
The call that triggers the error is updateGitlabCommitStatus and the status is verified to be one of https://github.com/jenkinsci/gitlab-plugin/blob/master/src/main/java/com/dabsquared/gitlabjenkins/gitlab/api/model/BuildState.java
As mentioned in the title, this is an "intermittent" problem, so reproduction is pretty much hit or miss. The reason for raising this issue is to hopefully get some ideas about why it might be happening and how to workaround or even get a solution - eg. do we need a later version of the plugin ?
Expected Results
updateGitlabCommitStatus works all of the time.
Actual Results
2023-04-27 21:13:18.035+0000 [id=67297737] SEVERE c.d.g.util.CommitStatusUpdater#updateCommitStatus: Failed to update Gitlab commit status for project '2392' javax.ws.rs.BadRequestException: HTTP 400 Bad Request at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.handleErrorStatus(ClientInvocation.java:241) at org.jboss.resteasy.client.jaxrs.internal.proxy.extractors.DefaultEntityExtractorFactory$3.extractEntity(DefaultEntityExtractorFactory.java:50) at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invokeSync(ClientInvoker.java:154) at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invoke(ClientInvoker.java:115) at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientProxy.invoke(ClientProxy.java:76) at com.dabsquared.gitlabjenkins.gitlab.api.impl.$Proxy209.changeBuildStatus(Unknown Source) at com.dabsquared.gitlabjenkins.gitlab.api.impl.ResteasyGitLabClient.changeBuildStatus(ResteasyGitLabClient.java:110) at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$13.execute(AutodetectingGitLabClient.java:177) at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$13.execute(AutodetectingGitLabClient.java:174) at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$GitLabOperation.execute(AutodetectingGitLabClient.java:397) at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient$GitLabOperation.access$100(AutodetectingGitLabClient.java:394) at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient.execute(AutodetectingGitLabClient.java:391) at com.dabsquared.gitlabjenkins.gitlab.api.impl.AutodetectingGitLabClient.changeBuildStatus(AutodetectingGitLabClient.java:173) at com.dabsquared.gitlabjenkins.util.CommitStatusUpdater.updateCommitStatus(CommitStatusUpdater.java:85) at com.dabsquared.gitlabjenkins.util.CommitStatusUpdater.updateCommitStatus(CommitStatusUpdater.java:97) at com.dabsquared.gitlabjenkins.workflow.GitLabBuildsStep$GitLabBuildStepExecution$1.onStart(GitLabBuildsStep.java:87) at org.jenkinsci.plugins.workflow.cps.CpsBodyExecution.launch(CpsBodyExecution.java:130) at org.jenkinsci.plugins.workflow.cps.CpsBodyInvoker.launch(CpsBodyInvoker.java:188) at org.jenkinsci.plugins.workflow.cps.DSL$ThreadTaskImpl.lambda$invokeBodiesAndSwitchToAsyncMode$0(DSL.java:784) at org.jenkinsci.plugins.workflow.cps.CpsStepContext.withBodyInvokers(CpsStepContext.java:538) at org.jenkinsci.plugins.workflow.cps.DSL$ThreadTaskImpl.invokeBodiesAndSwitchToAsyncMode(DSL.java:774) at org.jenkinsci.plugins.workflow.cps.DSL$ThreadTaskImpl.eval(DSL.java:739) at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:198) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:420) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$400(CpsThreadGroup.java:95) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:330) at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:294) at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:67) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source)
Anything else?
No response
Update: We have narrowed the issue to calling updateGitlabCommitStatus with the "same" state again or setting the state to a previous state (eg. from running to pending) . Is doing this invalid - i.e. cannot call with the same state multiple times or call setting the state back to a previous state ? Is this documented somewhere ?
@dg424 I think for now the best documentation is the code itself unfortunately, but there is incentive to improve our documentation this summer as a relevant Google Summer of Code (GSoC) 2023 project is under way. According to your reasoning the problem could be in how this GitLabBuildsStep.java file is written. So this might indeed be a faulty business logic issue. I will have more time to look into it later. However, if you have found a way to patch this bug meanwhile, by all means a pull request would be more than welcome!
