wdio-video-reporter icon indicating copy to clipboard operation
wdio-video-reporter copied to clipboard

Record videos per file/suite

Open davenh0000 opened this issue 3 years ago • 3 comments

Is it possible to record the video per suite/module/file instead of per test case?

davenh0000 avatar Jun 29 '21 15:06 davenh0000

Afaik the current behavior is that the video shows a complete session which contains all tests within a single file. Do you experience a different behavior?

christian-bromann avatar Jul 12 '21 10:07 christian-bromann

@christian-bromann No, its recording one video per it-statement or cucumber scenario, i.e. test case.

There have been a couple of requests for this functionality, also a similar one to have only one long video of all tests for each browser.

I started to work on the functionality but never finished.

A few issues to deal with:

  • Should it record all tests and override the saveAllTest option, or just the failing ones?
  • How do we let the user know where error have occurred in the video or what test is being run right now?
    • Some tests fail early and are super short
    • Some tests do almost the same thing to test different edge cases, if one fails early, how do you know which one you are watching?
      • Maybe we need to render the test name into the video with ffmpeg?
  • How to handle nestled suites?
    • Do we always make a video per suite, and let it contain all the included suites? Or do we only make a video of the top one?
        describe('1', () => {
          describe('2', () => {
            describe('3', () => {
      
              it('', () => {
      
              });
            });
      
            it('', () => {
              // May or may not be a test here
            });
          });
      
          it('', () => {
            // May or may not be a test here
          });
        });
      
  • Do we know for a fact that tests are always run in the order they are written?
  • What is the equivalent for cucumber? I dont know cucumber, but having options thats works for one framework and not the other feels iffy, so we need to make sure works in a similar way.
  • How do we make it compatible with Allure?
    • Maybe we should keep both individual test videos and also the suite ones?
  • Maybe we could get around most of it by offering an example on how to join videos in order with ffmpeg as post-processing?
  • And lastly .. Even though I guess opinions may differ among devs, I think having tests that depend on each other is a code smell, since it can lead to a cascade of failures. Not sure we should encourage it. But thats just me.

As Im typing this, I see that a post processor that joins only the top most suite would be the easiest way to do it. It wont solve the second issue above, but maybe it would be enough?

Any thoughts?

presidenten avatar Jul 12 '21 12:07 presidenten

Is https://github.com/webdriverio-community/wdio-video-reporter/issues/51 and this issue a duplicate? Maybe only @christian-bromann

Should it record all tests and override the saveAllTest option, or just the failing ones?

Can it not respect the settings applied already in regards to this?

How to handle nestled suites?

I would always do a video per suite, even if it means duplication, at least as a start

As Im typing this, I see that a post processor that joins only the top most suite would be the easiest way to do it. It wont solve the second issue above, but maybe it would be enough?

I would like to see a built-in solution without having to look for a post-processor.

I wonder if @ylilarry's solution is good enough for now?

jasonwilliams avatar Oct 01 '21 14:10 jasonwilliams

Is #51 and this issue a duplicate? Maybe only @christian-bromann

Yes, closing this one, thanks for all your comments.

duplicated of #51

christian-bromann avatar Jan 03 '24 03:01 christian-bromann

duplicated of https://github.com/webdriverio-community/wdio-video-reporter/issues/51

christian-bromann avatar Jan 03 '24 03:01 christian-bromann