cypress-testrail icon indicating copy to clipboard operation
cypress-testrail copied to clipboard

ignorePending not excluding pending test cases from testrail test run results

Open chrisgray37 opened this issue 9 months ago • 2 comments

I am using this in conjunction with cypress/grep. All the pending test cases are because they were skipped due to tagging. However, they are still showing up as untested in testrail run.

I am on this version of cypress-testrail:
"cypress-testrail": "^2.10.0",

Here is my cypress.config.js:

const {defineConfig} = require('cypress');
const TestRailReporter = require('cypress-testrail');

module.exports = defineConfig({
    video: true,
    reporter: 'cypress-multi-reporters',
    reporterOptions: {
        configFile: 'reporter-config.json',
    },
    scrollIntoView: {
        offset: {
            top: -150,
        },
    },
    viewportWidth: 1800,
    viewportHeight: 1000,
    chromeWebSecurity: false,
    e2e: {
        setupNodeEvents(on, config) {
            new TestRailReporter(on, config).register();
            require('@cypress/grep/src/plugin')(config);
            return config
        },
    },
    env: {
        grepFilterSpecs: true,
        grep: {
            showPending: true,
            showSkipped: true,
        }
    },
});

require('@applitools/eyes-cypress')(module);

And here is my testrail section of the cypress.env.json:

  "testrail": {
      "domain": "example.testrail.io",
      "username": "[email protected]",
      "password": "examplePW",
      "projectId": "42",
      "milestoneId": "",
      "suiteId": "",
      "runName": "Cypress Automation Pipeline Run: (__datetime__)",
      "ignorePending": true,
      "closeRun": true,
      "screenshots": true
  },

And here is my console when running the tests:

 1 passing (3s)
  16 pending

 Updating TestRail run R7088. Adding test cases: 9425, 9426, 9427, 9428, 9438, 9429, 9430, 9431, 9432, 9433, 9434, 9435, 9437, 9551, 9552, 9553, 9436 
 TestRun updated in TestRail: 7088 
 Ignoring pending test: C9425: Test Actuator Info 
 Ignoring pending test: C9426: Test Actuator Health 
 Ignoring pending test: C9427: Test Copy Test Data 
 Ignoring pending test: C9428: Test Generate, Reset, Then Delete Test Data 
 Ignoring pending test: C9438: Test Extract 
 Ignoring pending test: C9429: Test Details 
 Ignoring pending test: C9430: Test Resume 
 Ignoring pending test: C9432: Test Restore Collection 
 Ignoring pending test: C9433: Test Move Assets 
 Ignoring pending test: C9434: Test Delete Job 
 Ignoring pending test: C9435: Test List Jobs 
 Ignoring pending test: C9437: Test Find Job 
 Ignoring pending test: C9551: Test Delete Asset - Bad collectionName 
 Ignoring pending test: C9552: Test Delete Asset - Bad profileName 
 Ignoring pending test: C9553: Test Delete DFC 
 Ignoring pending test: C9436: FFV E2E 
 TestRail >> Sending case results to run R7088: C9431 
 Results sent to TestRail R7088 for: C9431 

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        17                                                                               │
  │ Passing:      1                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      16                                                                               │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     2 seconds                                                                        │
  │ Spec Ran:     api-ffv.cy.js                                                                    │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

chrisgray37 avatar May 24 '24 19:05 chrisgray37