gitlab-plugin icon indicating copy to clipboard operation
gitlab-plugin copied to clipboard

Skipped stages due to earlier failure keep 'pending' and cause the pipeline in 'running' status

Open crusoexia opened this issue 5 years ago • 7 comments

Issue

Skipped stages due to earlier failure keep pending and the pipeline status remain in running status.

Context

  • Gitlab plugin version: 1.5.12
  • Gitlab version: 9.0.2-ee
  • Jenkins version: 2.138.2
  • Job type: declarative pipeline.

Logs & Traces

Jenkins logs:

Test Suites: 1 failed, 80 passed, 81 total
Tests:       1 failed, 460 passed, 461 total
Snapshots:   40 passed, 40 total
Time:        73.369s
Ran all test suites.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] test:report: `node ../../scripts/test.js --env=jsdom --config config/jest/config.ci.js --coverage`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] test:report script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /var/lib/jenkins/.npm/_logs/2019-05-23T05_00_20_986Z-debug.log
lerna ERR! npm run test:report exited 1 in 'email-editor'
lerna ERR! npm run test:report exited 1 in 'email-editor'
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] test: `lerna exec -- npm run test:report`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /var/lib/jenkins/.npm/_logs/2019-05-23T05_00_21_013Z-debug.log
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // gitlabCommitStatus
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build)
Stage 'Build' skipped due to earlier failure(s)
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] deleteDir
[Pipeline] updateGitlabCommitStatus
[Pipeline] emailext
Not sending mail to unregistered user <xxxx>
Sending email to: <xxx>
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
There are still pending GitLab builds. Please check your configuration
[Pipeline] // gitlabBuilds
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE

GitLab pipeline status: Screen Shot 2019-05-23 at 1 50 50 PM

Problem description

Just trying to update each stages' status to gitlab, but found when there is early stage failed, all subsequent stage will keep in 'pending' status, and the whole pipeline status remains in running

Jenkins File

def sourceBranch = env.gitlabBranch ?: params.gitlabBranch

pipeline {
  agent any
  environment {
    projectName = "<project-name>"
    checkoutDir = "<dir>"
    repoSSHURL = "<git-url>"
    mailTo = "<email>"
  }
  parameters {
    string(
      name: "gitlabBranch",
      defaultValue: "master",
      description: "Branch to build"
    )
  }
  options {
    skipDefaultCheckout()
    gitLabConnection("Gitlab")
    gitlabBuilds(builds: ["Init", "Inspect", "Test", "Build"])
  }
  triggers {
    gitlab(
      triggerOnPush: false,
      triggerOnMergeRequest: true,
      skipWorkInProgressMergeRequest: true,
      branchFilterType: "All",
      secretToken: "<token>"
    )
  }
  stages {
    stage("Checkout") {
      steps {
        script {
          echo "Gitlab merge request triggered build"
          sourceBranch = env.gitlabSourceBranch
          currentBuild.displayName = env.gitlabSourceBranch
          checkout ([
            $class: "GitSCM",
            branches: [[name: "origin/${env.gitlabSourceBranch}"]],
            extensions: [
              [$class: "RelativeTargetDirectory", relativeTargetDir: "${checkoutDir}"],
              [$class: "LocalBranch"],
              [$class: "PreBuildMerge", options: [
                fastForwardMode: "FF",
                mergeRemote: "origin",
                mergeStrategy: "DEFAULT", 
                mergeTarget: env.gitlabTargetBranch
              ]]
            ],
            userRemoteConfigs: [[url: "${repoSSHURL}"]]
          ])
        }
      }
    }
    stage("Init") {
      steps {
        gitlabCommitStatus("Init") {
          dir(checkoutDir) {
            sh "npm ci"
            sh "npm run bootstrap"
            sh "npm run setup"
          }
        }
      }
    }
    stage("Inspect") {
      steps {
        gitlabCommitStatus("Inspect") {
          dir(checkoutDir) {
            sh "npm run inspect"
          }
        }
      }
    }
    stage("Test") {
      steps {
        gitlabCommitStatus("Test") {
          dir(checkoutDir) {
            sh "npm run test"
          }
        }
      }
    }
    stage("Build") {
      steps {
        gitlabCommitStatus("Build") {
          dir(checkoutDir) {
            sh "npm run build"
          }
        }
      }
    }
  }
  post {
    failure {
      emailext([
          body: "${projectName} build failed, see: ${env.BUILD_URL}",
          mimeType: "text/html",
          replyTo: "<reply-email>",
          recipientProviders: [[$class: "CulpritsRecipientProvider"]],
          subject: "${projectName} build (${sourceBranch}) - Failure",
          to: "${mailTo}"
      ])
    }
    always {
      deleteDir()
    }
  }
}

crusoexia avatar May 23 '19 06:05 crusoexia

I've also encountered this issue. Is there a current solution?

I could not find a post action + notification combination that I can use to mark the build status done for the stages that are skipped.

ieugen avatar Jan 03 '21 09:01 ieugen

I can think of some solutions to solve this:

  1. Provide a post action step that triggers when a stage is skipped - so we can update gitlab status when a stage is skipped https://www.jenkins.io/doc/book/pipeline/syntax/#post . This should be provided upstream IMO.

  2. Provide a way to add build steps when entering a new stage instead of defining all of them up-front. This is less ideal IMO.

  3. Set all the gitlabBuilds initially and have a way to get the skipped stages from Jenkins. Add a post action at the pipeline level that marks all the skipped stages as done.

I think solution 1 has value for other people as well. If anyone knows where to report the issue I can do it. It's not clear to me where post build actions are developed.

ieugen avatar Jan 03 '21 09:01 ieugen

any update on this? I've the same behavior

leandrofrs avatar Apr 27 '21 11:04 leandrofrs

I've also stuck with this problem.... Could you please fix it?

ragumix avatar Oct 28 '21 16:10 ragumix

any update here ? I've the same behavior

xpicio avatar Dec 22 '21 14:12 xpicio