cypress-testrail-simple icon indicating copy to clipboard operation
cypress-testrail-simple copied to clipboard

[Github Actions] TestRail TC's from test run are not marked as passed/failed

Open adiqa opened this issue 2 years ago โ€ข 1 comments

I've configured my TestRail to work with Cypress and tests are running fine.

Also I've provided all of those TC ID's in Cypress tests, but whenever the run is finished, that testrun remains opened and testcases are nor passed nor failed.

Below is mine Github Actions workflow:

name: 'CI'
on: [deployment_status]
jobs:
  e2e:
    if: github.event.deployment_status.state == 'success' && github.event.deployment.environment == 'preview'
    runs-on: ubuntu-latest
    env:
      # pass TestRail settings from the project secrets
      # via environment variables
      TESTRAIL_HOST: ${{secrets.TESTRAIL_HOST}}
      TESTRAIL_USERNAME: ${{secrets.TESTRAIL_USERNAME}}
      TESTRAIL_PASSWORD: ${{secrets.TESTRAIL_PASSWORD}}
      # GITHUB_TOKEN for other stuff
      GITHUB_TOKEN: ${{ secrets.GA_TOKEN }}
      # the project ID is not that secret
      TESTRAIL_PROJECTID: 5
      # the Suite ID is not that secret
      TESTRAIL_SUITEID: S76

    steps:
      - name: Dump GitHub context
        env:
          GITHUB_CONTEXT: ${{ toJSON(github) }}
        run: echo "$GITHUB_CONTEXT"

      - name: Checkout ๐Ÿ›Ž
        uses: actions/checkout@v2

      # Install NPM dependencies, cache them correctly
      - name: Install Cypress ๐Ÿ“ฆ
        uses: cypress-io/github-action@v4
        with:
          runTests: false

      # verify that all case IDs in the spec files are valid
      # for this TestRail project
      - name: Check case IDs ๐Ÿ”Ž
        run: |
          runId=$(npx testrail-check-case-ids --find-specs)
        env:
          DEBUG: cypress-testrail-simple

      # you can pass GitHub information in the name and description
      # to include in the TestRail run information
      - name: Start TestRail Run ๐Ÿƒ๐Ÿปโ€โ™‚๏ธ
        id: testRail
        run: |
          commitSubject="${{ github.event.commits[0].message }}"
          runName="Testing on GitHub Actions: ${commitSubject}"
          runDescription="Cypress tests for commit ${GITHUB_SHA} ${GITHUB_REF}"
          echo ${commitSubject}
          echo ${runName}
          echo ${runDescription}
          runId=$(npx testrail-start-run "${runName}" "${runDescription}")
          echo "TestRail run id ${runId}"
          # save the run ID as the output from this step
          echo "runId=${runId}" >> $GITHUB_OUTPUT
        env:
          DEBUG: cypress-testrail-simple

      # Now run the Cypress tests, and the cypress-testrail-simple
      # plugin will automatically add the results to TestRail
      - name: Cypress tests ๐Ÿงช
        uses: cypress-io/github-action@v4
        with:
          runTests: true
          record: true
          install-command: echo "Already installed"

        env:
          # pass the run ID from the previous step via an env variable
          TESTRAIL_RUN_ID: ${{ steps.testRail.outputs.runId }}
          # pass the Dashboard record key as an environment variable
          CYPRESS_RECORD_KEY: ${{secrets.CYPRESS_RECORD_KEY}}
          CYPRESS_RETRIES: 1
          DEBUG: cypress-testrail-simple

      - name: Print TestRail run results ๐Ÿ“‹
        if: ${{ always() }}
        run: |
          npx testrail-run-results --run ${{ steps.testRail.outputs.runId }}
        env:
          DEBUG: cypress-testrail-simple

      - name: Close TestRail Run ๐Ÿ
        # always close the test run, even if the previous steps have failed
        if: ${{ always() }}
        run: |
          npx testrail-close-run ${{ steps.testRail.outputs.runId }}
        # Upload recording as a proof
      - uses: actions/upload-artifact@v2
        if: failure()
        with:
          retention-days: 3
          path: |
            cypress/videos

adiqa avatar Oct 27 '22 16:10 adiqa