app icon indicating copy to clipboard operation
app copied to clipboard
verified publisher icon testomatio

The total number of tests shown in the Groups Report, and combined report is not correct

Open SusanaAlves opened this issue 1 year ago • 12 comments

Steps to reproduce:

  1. Have some manual runs previously run.
  2. Create a group for a given release and add those existing runs.
  3. On Groups, click on the folder with your release. You will see that the number of tests passed is 0 instead of the sum of all the tests ran and passed. Screenshot 2024-08-26 at 15 47 39
  4. Click on the Combined Report button. The totals show 0. Screenshot 2024-08-26 at 15 49 04

SusanaAlves avatar Aug 26 '24 15:08 SusanaAlves

Hello, any update on this?

SusanaAlves avatar Sep 11 '24 15:09 SusanaAlves

Hi @SusanaAlves! The fix should be live now. Could you please check it on your side?

TetianaKhomenko avatar Sep 12 '24 06:09 TetianaKhomenko

Fixed here: Screenshot 2024-09-12 at 12 51 16

Not fixed in the combined report. The totals are still 0. Screenshot 2024-09-12 at 12 52 08

SusanaAlves avatar Sep 12 '24 11:09 SusanaAlves

@TetianaKhomenko, the combined report has not been fixed. The totals are showing 0.

SusanaAlves avatar Sep 12 '24 11:09 SusanaAlves

@SusanaAlves it's hard to understand the problem with small pieces of screen. You can share the whole screen screenshot here or in our Slack private channel

TetianaKhomenko avatar Sep 12 '24 12:09 TetianaKhomenko

@TetianaKhomenko, here is a full screenshot: Screenshot 2024-09-12 at 15 37 18

Can you see that all the totals are 0 instead of showing the correct number of test cases in each status?

SusanaAlves avatar Sep 12 '24 14:09 SusanaAlves

@SusanaAlves could you please show this run

CleanShot 2024-09-12 at 20 06 18@2x

Overall I have the results. I assume it can be because test statuses are not shown for the second run.

Here is my example:

CleanShot 2024-09-12 at 20 04 31@2x

TetianaKhomenko avatar Sep 12 '24 18:09 TetianaKhomenko

I have these data: Screenshot 2024-09-13 at 13 31 46

Screenshot 2024-09-13 at 13 32 26

When I click on Combined report I can't see the totals being updated, even if I change the main Run: Screenshot 2024-09-13 at 13 33 13

Screenshot 2024-09-13 at 13 34 17

SusanaAlves avatar Sep 13 '24 12:09 SusanaAlves

@SusanaAlves Do you use branches? Or may be tests have different versions?

Please share screenshots both of runs on Report page Image

littlefrontender avatar Sep 18 '24 13:09 littlefrontender

@SusanaAlves Your runs have different tests, none of them match, that's why the counters show 0.

littlefrontender avatar Sep 18 '24 14:09 littlefrontender

@littlefrontender, that's true. The tests are different, but I thought a combined report would show all the totals. If I create a Group to group all the runs performed for a given release, it's very likely that the several runs will have different test cases, as we are testing new features and new tests were created. I thought that with the Combined report, I could see all the totals. Is that not possible, then? So, does the combined report just work if we have runs with the same tests?

SusanaAlves avatar Sep 18 '24 17:09 SusanaAlves

Hi @SusanaAlves! Thanks for your explanation. Originally when Combined Report was designed it was meant to compare test results, but not combine. So please feel free to create a feature request on your suggestion.

TetianaKhomenko avatar Oct 01 '24 10:10 TetianaKhomenko