allure2 icon indicating copy to clipboard operation
allure2 copied to clipboard

Displayed `Total` suite duration is significantly higher than `Sum` suite duration

Open imaxymiv opened this issue 3 years ago • 9 comments

The report has Total and Sum indicators that represent suite duration. Sum represent the proper value, but Total shows extra time. I checked the setup and tear down sections. They didn't take long compared to what is displayed in Total. Please, check the screenshot.

To Reproduce

  1. Go to 'Suites' section
  2. Click on 'Toggle groups info'
  3. Check the duration of the suite

Expected behavior Total should represent Sum + time for setup and tear down.

Screenshots image

Environment (please complete the following information):

Allure version 2.18.1
Test framework [email protected]
Allure adaptor [email protected]
Generate report using [email protected]

Additional context For some reason, there are 8+ hours added to the duration.

imaxymiv avatar Jul 22 '22 13:07 imaxymiv

Total is the time between start of the very first test and end of the very last one Sum is the sum of all tests durations Max is the max of all tests durations

baev avatar Jul 22 '22 13:07 baev

@baev could you clarify if Sum includes retries?

imaxymiv avatar Jul 22 '22 14:07 imaxymiv

only results that visible in group. Usually retries are hidden, so the answer is not

baev avatar Jul 22 '22 14:07 baev

So such a huge difference can be because of retries, setups, and tear-downs only. Correct me if I'm wrong.

imaxymiv avatar Jul 22 '22 14:07 imaxymiv

Nope:

Feature 1
  T1
  T2
Feature 2
  T3
  T4

if you run tests in single thread and the order is the following T1 -> T3 -> T2 -> T4 and there is no gaps between tests and durations are the following: T1 - 1m , T2 - 5m , T3 - 1h , T4 - 1s then the following summary will be shown for groups:

Name Total Sum Max
Feature 1 1h 6m 6m 5m
Feature 2 1h 5m 1s 1h 1s 1h

and the timeline

A -------- B --------- C -------- D -------- E
      1m         1h          5m          1s
 
A - start of T1
B - end of T1 and start of T3
C - end of T3 and start of T2
D - end  of T2 and start of T4
E - end of T4

Total for Feature 1 is time between start of T1 and end of T2

start of T1 = X end of T2 = X + AB + BC + CD = X + 1m + 1h + 5m total = X + 1m + 1h + 5m - X = 1h 6m

baev avatar Jul 22 '22 14:07 baev

Thanks for the answer. In case we have multiple everything running in parallel, will Total be equal Sum?

imaxymiv avatar Jul 26 '22 04:07 imaxymiv

@baev could you clarify?

imaxymiv avatar Aug 10 '22 06:08 imaxymiv

@baev any updates on this?

imaxymiv avatar Sep 06 '22 14:09 imaxymiv

Thanks for the answer. In case we have multiple everything running in parallel, will Total be equal Sum?

nope, see https://github.com/allure-framework/allure2/issues/1704#issuecomment-1192601120

baev avatar Sep 06 '22 14:09 baev