External job is added to the wrong pipeline

[danielvijge]

This required Gitlab 11.6+ which supports the only: merge_requests syntax in the CI/CD pipeline.

Description I want to set up a flow where the master branch is protected, development happens in feature branches, and everything is merged via merge requests in Gitlab. To achieve this, I want to run a preview analysis on every commit to a feature branch, so that comments are added to the commit. When there is a merge request, I want to run a full analysis, so that Sonarqube checks the quality gate, and can determine if the merge request is allowed to be merged.

This is a slightly modified configuration of the one mentioned in the readme:

sonarqube_master_job:
  stage: test
  only:
    - merge_requests
  script:
    - mvn --batch-mode verify sonar:sonar -Dsonar.host.url=$SONAR_URL -Dsonar.login=$SONAR_LOGIN -Dsonar.gitlab.project_id=$CI_PROJECT_PATH -Dsonar.gitlab.commit_sha=$CI_COMMIT_SHA -Dsonar.gitlab.ref_name=$CI_COMMIT_REF_NAME

sonarqube_preview_feature_job:
  stage: test
  only:
    - /^feature\/*/
  script:
    - git checkout origin/master
    - git merge $CI_COMMIT_SHA --no-commit --no-ff
    - mvn --batch-mode verify sonar:sonar -Dsonar.host.url=$SONAR_URL -Dsonar.login=$SONAR_LOGIN -Dsonar.analysis.mode=preview -Dsonar.gitlab.project_id=$CI_PROJECT_PATH -Dsonar.gitlab.commit_sha=$CI_COMMIT_SHA -Dsonar.gitlab.ref_name=$CI_COMMIT_REF_NAME

The first time something is pushed to a feature branch, the preview analysis runs. This works as expected. Now when a merge request is created, the quality gate analysis runs. This also works as expected. However, when there is an additional commit to the same feature branch that already has an open merge request, GitLab starts both the sonarqube_master_job and the sonarqube_preview_feature_job. In general, this is OK, because it's new code, and I want both the comments and the quality gate.

In my tests, the sonarqube_preview_feature_job is created with pipeline ID 1, the sonarqube_master_job is created with pipeline ID 2. Pipeline 1 finishes first, but the external sonarqube step is added to pipeline 2. After pipeline 2 finishes, the external job is added to pipeline 2 as well. The external job for pipeline 1 is visible in pipeline 2 as a retried job.

My guess is that this is happening because only the CI_COMMIT_SHA and CI_COMMIT_REF_NAME are passed. But these are the same for both pipelines. No CI_PIPELINE_ID is passes, that tells the plugin to add it to the correct pipeline.

Jobs for the sonarqube_preview_feature_job:

Jobs for the sonarqube_master_job:

[danielvijge]

It's actually also possible to recreate this without merge requests, but with the only: web feature when running a manual pipeline. For example:

sonarqube_master_job:
  stage: test
  only:
    -  master
  script:
    - mvn --batch-mode verify sonar:sonar -Dsonar.host.url=$SONAR_URL -Dsonar.login=$SONAR_LOGIN -Dsonar.gitlab.project_id=$CI_PROJECT_PATH -Dsonar.gitlab.commit_sha=$CI_COMMIT_SHA -Dsonar.gitlab.ref_name=$CI_COMMIT_REF_NAME

sonarqube_preview_feature_job:
  stage: test
  only:
    - /^feature\/*/
  script:
    - git checkout origin/master
    - git merge $CI_COMMIT_SHA --no-commit --no-ff
    - mvn --batch-mode verify sonar:sonar -Dsonar.host.url=$SONAR_URL -Dsonar.login=$SONAR_LOGIN -Dsonar.analysis.mode=preview -Dsonar.gitlab.project_id=$CI_PROJECT_PATH -Dsonar.gitlab.commit_sha=$CI_COMMIT_SHA -Dsonar.gitlab.ref_name=$CI_COMMIT_REF_NAME

deploy:
  stage: deploy
  only:
  - web
  script:
  - do_deployment

When manually running a pipeline on a branch that has already an active pipeline with a Sonarqube check, the results is that the external stage is added to the deploy pipeline (even though that pipeline doesn't have a Sonarqube job at all.

[Hellevar]

Try to specify 'sonar.gitlab.failure_notification_mode' with 'exit-code' value (default is 'commit-status')

[danielvijge]

From the documentation it is not very clear what failure_notification_mode=exit-code does exactly. I set this, and now it doesn't create the External stage is not created at all. I kind of like this stage, because it gives a quick overview on Gitlab's Pipelines screen about the status of either a preview or analysis

[t-buss]

@danielvijge Any updates on this? Seem to have something similar in our setup. We also want to protect master and get the analysis in the MRs before merging. Now, we have two pipelines. Also, it seems to be related to the only keyword. Before, we did not use it and everything worked great

[pierrefevrier]

Hi,

I noticed the same as @t-buss: 1 push on MR = 2 pipelines with the use of only: merge_requests.

What I expect is sonar analysis job to add it's results on the first pipeline, not on a new one.
Here is my Sonar job configuration:

code-quality:
  stage: test
  image: $DOCKER_MAVEN_IMAGE
  script:
    - mvn $MAVEN_CLI_OPTS 
      org.jacoco:jacoco-maven-plugin:prepare-agent 
      package 
      sonar:sonar
      --define maven.test.failure.ignore=true 
      --define revision=$CI_COMMIT_REF_NAME
      --define sonar.analysis.mode=preview 
      --define sonar.gitlab.commit_sha=$CI_COMMIT_SHA 
      --define sonar.gitlab.ref_name=$CI_COMMIT_REF_NAME 
      --define sonar.gitlab.project_id=$CI_PROJECT_ID
  only:
    - merge_requests

[gihif]

Hi guys, @danielvijge @pierrefevrier I also had a problem like that before. then I first learn the gitlab API about commit status here and it turns out that this bug problem can still be solved by changing the value of the sonar.gitlab.ref_name variable.

so, if you use

  only:
    - branch_name

then the value used is

/d:sonar.gitlab.ref_name="${CI_COMMIT_REF_NAME}"

but, if you use

  only:
    - merge_requests

then the value used is

/d:sonar.gitlab.ref_name="${CI_MERGE_REQUEST_REF_PATH}"

here my code

RUN dotnet-sonarscanner begin \
        /k:"${SONAR_AUTH_KEY}" \
        /n:"${SONAR_PROJ_NAME}" \
        /d:sonar.host.url="${SONAR_HOST_URL}" \
        /d:sonar.login="${SONAR_AUTH_TOKEN}" \
        /d:sonar.verbose=false \
        /d:sonar.analysis.mode=preview \
        /d:sonar.gitlab.commit_sha="${CI_COMMIT_SHA}" \
        /d:sonar.gitlab.ref_name="${CI_MERGE_REQUEST_REF_PATH}" \
        /d:sonar.gitlab.project_id="${CI_PROJECT_ID}" \
        /d:sonar.gitlab.all_issues="${CI_ALLISSUE_REPORT}" \
        /d:sonar.gitlab.ci_merge_request_iid="${CI_MERGE_REQUEST_IID}" \
        /d:sonar.gitlab.merge_request_discussion=true \
        && \
    dotnet build ${DOTNET_SLN_NAME}.sln && \
    dotnet-sonarscanner end /d:sonar.login="${SONAR_AUTH_TOKEN}"

the result before fix the result after fix

CMIIW... :)