cypress-testrail icon indicating copy to clipboard operation
cypress-testrail copied to clipboard

My Testrail Results are being sent as Failed and Passed at the same time

Open GregJacobs82 opened this issue 1 year ago • 2 comments

 2023-10-04 at 1 54 18 PM

NOTE: I did just recently update to latest cypress-testrail v2.8.1

Notice the identical timestamps in the Testrail report. Any thoughts as to why it is sending both a passing and failing status? It SHOULD just be sending the failed status (in this specific case).


I think it might be due to my test scenario is a Scenario Outline: with more than 1 example to run. In this test case scenario outline, the First test example for USA fails, but the second test example for CANADA passes. If any example fails in the scenario outline, it should fail the test:

────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  PublicUser/Homepage.feature                                                     (2 of 2)
  Estimated: 1 minute, 1 second


  Public User > Homepage
    (Attempt 1 of 2) C305363: My test scenario outline [USA] (example #1)
    1) C305363: My test scenario outline [USA] (example #1)
    ✓ C305363: My test scenario outline [Canada] (example #2) (6040ms)


  1 passing (1m)
  1 failing

  1) Public User > Homepage
       C305363: My test scenario outline [USA] (example #1):
     AssertionError: Timed out retrying after 16000ms: expected 'https://xxxxx.com' to include 'xxxx.expected'
      at Context.eval (cypress/integration/cucumber/PublicUser/common/common.spec.js:392:21)
      at Context.resolveAndRunStepDefinition (node_modules/cypress-cucumber-preprocessor/lib/resolveStepDefinition.js:193:0)
      at Context.eval (node_modules/cypress-cucumber-preprocessor/lib/createTestFromScenario.js:27:0)



  TestRail >> Sending case results to run R651: C305363,C305363 
  Results sent to TestRail for: C305363,C305363 

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        2                                                                                │
  │ Passing:      1                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  2                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     1 minute, 11 seconds                                                             │
  │ Estimated:    1 minute, 1 second                                                               │
  │ Spec Ran:     PublicUser/Homepage.feature                                                      │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

Any help is greatly appreciated. Thank you!

GregJacobs82 avatar Oct 04 '23 18:10 GregJacobs82