Pipeline id implicitly reused for different webhooks/jobs on same branch commit

[cjquintero]

Version report

Jenkins and plugins versions report:

Jenkins: 2.263.4 on Windows
Gitlab-plugin: 1.5.13
GitLab version: 13.8.1-ee

Scenario:

Suppose that you want:

• a CI-pipeline to validate a merge request while open (stages: Build, Test)
• a CD-pipeline to deploy artifacts when the merge request is accepted (stages: Deploy)

You create two pipelines in Jenkins:

• The CI-pipeline will be filtered in its Jenkins configuration to allow only "Opened merge request events"
• The CD-pipeline will be filtered in its Jenkins configuration to allow only "Accepted merge request events"

You create two webhooks in GitLab for the repo. Both of them are triggered by "merge request events".

Reproduction steps

In Jenkins:

• Create a "CI-Pipeline" with code like this:

pipeline {  
    agent any
    options {
        gitLabConnection('mygitlabconnection')
        gitlabBuilds(builds: ['Build', 'Test'])
    }
    stages {
        stage("Build") {
            steps {
                gitlabCommitStatus(name: 'Build') {
                    echo "Build"
                }
            }
        }
        stage("Test") {
            steps {
                gitlabCommitStatus(name: 'Test') {
                    echo "Test"
                }
            }
        }
    }
}

• Check the checkboxes "Build when a change is pushed to GitLab..." and "Opened Merge Request Events"

• Create a "CD-pipeline" with code like this:

pipeline {  
    agent any
    options {
        gitLabConnection('mygitlabconnection')
        gitlabBuilds(builds: ['Deploy'])
    }
    stages {
        stage("Deploy") {
            steps {
                gitlabCommitStatus(name: 'Deploy') {
                    echo "Deploy"
                }
            }
        }
    }
}

• Check the checkboxes "Build when a change is pushed to GitLab..." and "Accepted Merge Request Events"

In GitLab:

• Create a repo with branch "master" with one file.
• Create branch "feature-1" from branch "master".
• In Settings > General, section "Merge request" > "Merge checks", check "Pipelines must succeed".
• In Settings > Webhooks, create two webhooks, one for the CI-pipeline and the other for the CD-pipeline, both triggered on "Merge Requests Events" (each one will filter in Jenkins the desired merge request event, as explained above).
• Modify the file in the branch "Feature-1" branch, create a commit and create a merge request from branch "Feature-1" to branch master".
• The CI-pipeline is launched in Jenkins as expected and an associated external pipeline is created in GitLab with pending GitLab jobs "Build" and "Test" (due to the gitlabBuilds step). The "Merge" button is not enabled until the CI-pipeline finishes:

• Click the "Merge" button.
• The CD-pipeline is launched in Jenkins as expected, so far so good, but...

Expected Result

Even if the commit is the same for the CI-pipeline and CD-pipeline, a new GitLab external pipeline should be created for the CD-pipeline of Jenkins with just one GitLab job ("Deploy").

Actual result

In GitLab the same external pipeline that was used for the CI-pipeline of Jenkins is now reused for the CD-pipeline of Jenkins; the GitLab job of the CD-pipeline ("Deploy") is added to it between the GitLab jobs "Build" and "Test" (because GitLab sorts them by name):

Suggested solution

Since the GitLab Commit Status API has a pipeline_id parameter (for "when multiple pipelines on the same commit SHA have been triggered", see https://docs.gitlab.com/ee/api/commits.html#post-the-build-status-to-a-commit), the gitLabBuilds step should create explicitly a GitLab pipeline using the GitLab Pipelines API, see https://docs.gitlab.com/ee/api/pipelines.html#create-a-new-pipeline, rather than relying on GitLab Commit Status API's logic to create a new pipeline or not, and use that pipeline id in subsequent gitlabCommitStatus steps.

Maybe just creating the pipeline in the GitLabBuilds step suffices (without modifying the gitlabCommitStatus step to pass the pipeline id) if GitLab uses the latest pipeline when several pipelines exist for the same commit (see code: https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/api/commit_statuses.rb)