Recording filenames should contain testclass information not only for surefire output folder

[danielFesenmeyer]

trafficstars

Description of the problem: When using the surefire output folder when running the tests, output files with the following naming pattern are created (in case recording of screenshot and video is enabled): testClassName>/testMethodName_args_sessionId.ending. For example: org.example.RecordingTest/testRecordingMethod_arg0_chrome_00daaec2ab555cbb1a8e98e573729959.mp4 What I like about this naming pattern is that it contains both the test-class-name and the test-method-name, which makes it quite easy to find the file(s) for a certain test.

When using a different output folder (for example, the failsafe output folder "./target/failsafe-reports/"), a different naming pattern is used: testMethodName_args_sessionId.ending. For example: testRecordingMethod_arg0_chrome_00daaec2ab555cbb1a8e98e573729959.mp4. As you can see, the test-class-name is not contained in the filename. This makes it difficult to find the correct video/screenshot in case different test classes define the same test method name.

What about just using the "surefire output pattern" for all output folders?

Browser and version: Latest version of Chrome in Docker.

Operating system: 20.04.2-Ubuntu, x86_64

Selenium-Jupiter version: 4.0.1

Selenium-Jupiter use:

package org.example;

import io.github.bonigarcia.seljup.DockerBrowser;
import io.github.bonigarcia.seljup.SeleniumJupiter;
import io.github.bonigarcia.seljup.config.Config;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.RegisterExtension;
import org.openqa.selenium.WebDriver;

import static io.github.bonigarcia.seljup.BrowserType.CHROME;
import static org.assertj.core.api.Assertions.assertThat;

class RecordingFailsafeOutputFolderTest {

    @RegisterExtension
    static SeleniumJupiter seleniumJupiter = new SeleniumJupiter();

    @BeforeAll
    static void setup() {
        Config config = seleniumJupiter.getConfig();
        config.enableScreenshot();
        config.enableRecording();
        config.setOutputFolder("./target/failsafe-reports/");
    }

    @Test
    void testRecordingMethod(@DockerBrowser(type = CHROME) WebDriver driver) {
        driver.navigate().to("https://bonigarcia.dev/selenium-webdriver-java/");

        assertThat(driver.getTitle()).contains("Selenium");
    }

}

Selenium-Jupiter traces:

19:22:26.686 [main] TRACE i.g.b.seljup.SeleniumJupiter - Resolving parameter org.openqa.selenium.WebDriver arg0 (contextId [engine:junit-jupiter]/[class:org.example.RecordingFailsafeOutputFolderTest], index 0)
19:22:26.694 [main] TRACE i.g.bonigarcia.seljup.config.Config - Property key sel.jup.selenium.server.url not found, using default value
19:22:26.696 [main] TRACE i.g.b.seljup.SeleniumJupiter - Single session false
19:22:27.671 [main] TRACE i.g.b.seljup.CapabilitiesHandler - Getting capabilities for type=interface org.openqa.selenium.WebDriver -- browserType=Optional[CHROME]
19:22:27.699 [main] TRACE i.g.b.seljup.CapabilitiesHandler - Getting options for class org.openqa.selenium.chrome.ChromeOptions
19:22:27.708 [main] TRACE i.g.b.seljup.CapabilitiesHandler - Exception reading preferences of class org.openqa.selenium.chrome.ChromeOptions (org.openqa.selenium.chrome.ChromeOptions.addPreference(java.lang.String, java.lang.Object))
19:22:27.709 [main] TRACE i.g.b.seljup.CapabilitiesHandler - Gathered Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}}
19:22:27.721 [main] TRACE i.g.bonigarcia.seljup.OutputHandler - Output folder ./target/failsafe-reports/
19:22:27.721 [main] TRACE i.g.b.seljup.SeleniumJupiter - Put manager io.github.bonigarcia.wdm.managers.ChromeDriverManager@6c0d7c83 in map (context id [engine:junit-jupiter]/[class:org.example.RecordingFailsafeOutputFolderTest])
19:22:27.721 [main] TRACE i.g.b.seljup.SeleniumJupiter - Adding io.github.bonigarcia.wdm.managers.ChromeDriverManager@6c0d7c83 to new map (id [engine:junit-jupiter]/[class:org.example.RecordingFailsafeOutputFolderTest])
19:22:27.730 [main] TRACE i.g.b.wdm.docker.DockerService - Docker image: selenoid/vnc:chrome_97.0
19:22:27.731 [main] DEBUG i.g.b.wdm.cache.ResolutionCache - Resolution chrome-container-latest=97.0 in cache (valid until 19:35:03 20/01/2022 CET)
19:22:27.732 [main] TRACE i.g.bonigarcia.wdm.config.Config - The memory size 256m is equivalent to 268435456 bytes)
19:22:27.735 [main] TRACE i.g.bonigarcia.wdm.config.Config - The memory size 128m is equivalent to 134217728 bytes)
19:22:27.741 [main] INFO  i.g.b.wdm.docker.DockerService - Starting Docker container selenoid/vnc:chrome_97.0
19:22:28.003 [main] TRACE i.g.b.wdm.docker.DockerService - Adding sysadmin capabilty
19:22:28.004 [main] TRACE i.g.b.wdm.docker.DockerService - Using shm size: 268435456
19:22:28.004 [main] TRACE i.g.b.wdm.docker.DockerService - Using network: bridge
19:22:28.004 [main] TRACE i.g.b.wdm.docker.DockerService - Using exposed ports: [4444]
19:22:28.006 [main] TRACE i.g.b.wdm.docker.DockerService - Using mounts: [Mount(type=TMPFS, source=null, target=/tmp, readOnly=null, bindOptions=null, volumeOptions=null, tmpfsOptions=TmpfsOptions(sizeBytes=134217728, mode=null))]
19:22:28.006 [main] TRACE i.g.b.wdm.docker.DockerService - Using envs: [TZ=Etc/UTC, LANG=EN, SCREEN_RESOLUTION=1280x1080x24]
19:22:29.079 [main] TRACE i.g.b.wdm.docker.DockerService - Port list {4444/tcp=[Lcom.github.dockerjava.api.model.Ports$Binding;@4c36250e} -- Exposed port 4444/tcp = [0.0.0.0:49269]
19:22:29.111 [main] TRACE i.g.b.wdm.docker.DockerService - Browser remote URL http://localhost:49269/
19:22:29.113 [main] TRACE i.g.bonigarcia.wdm.WebDriverManager - The Selenium Serverl URL is http://localhost:49269/
19:22:29.116 [main] DEBUG i.g.b.wdm.webdriver.WebDriverCreator - Creating WebDriver object for chrome at http://localhost:49269/ with Capabilities {browserName: chrome, goog:chromeOptions: {args: [--disable-gpu], extensions: []}}
19:22:29.128 [main] TRACE i.g.b.wdm.webdriver.WebDriverCreator - SocketException creating WebDriver object (Connection reset)
19:22:30.145 [main] TRACE i.g.b.wdm.webdriver.WebDriverCreator - Requesting http://localhost:49269/ (the response code is 404)
Jan 20, 2022 7:22:30 PM org.openqa.selenium.remote.tracing.opentelemetry.OpenTelemetryTracer createTracer
INFO: Using OpenTelemetry for tracing
Jan 20, 2022 7:22:32 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
19:22:32.100 [main] DEBUG i.g.b.wdm.webdriver.WebDriverCreator - The sessionId is 2a505763be0a9438d1d8e71c82bd830d
19:22:32.101 [main] DEBUG i.g.b.wdm.cache.ResolutionCache - Resolution recorder-container=7.1 in cache (valid until 19:35:10 20/01/2022 CET)
19:22:32.103 [main] TRACE i.g.bonigarcia.wdm.config.Config - The screen resolution 1280x1080x24 corresponds to a video size of 1280x1080
19:22:32.115 [main] INFO  i.g.b.wdm.docker.DockerService - Starting Docker container selenoid/video-recorder:7.1
19:22:32.115 [main] TRACE i.g.b.wdm.docker.DockerService - Adding sysadmin capabilty
19:22:32.115 [main] TRACE i.g.b.wdm.docker.DockerService - Using network: bridge
19:22:32.115 [main] TRACE i.g.b.wdm.docker.DockerService - Using binds: [/home/fed1imb/work/lab/seljupiter-tests/./target/failsafe-reports:/data:rw]
19:22:32.116 [main] TRACE i.g.b.wdm.docker.DockerService - Using envs: [BROWSER_CONTAINER_NAME=172.17.0.6, FILE_NAME=testRecordingMethod_arg0_chrome_2a505763be0a9438d1d8e71c82bd830d.mp4, VIDEO_SIZE=1280x1080, FRAME_RATE=12]
19:22:32.620 [main] INFO  i.g.bonigarcia.wdm.WebDriverManager - Starting recording ./target/failsafe-reports/testRecordingMethod_arg0_chrome_2a505763be0a9438d1d8e71c82bd830d.mp4
19:22:36.366 [main] TRACE i.g.bonigarcia.seljup.OutputHandler - Output folder ./target/failsafe-reports/
19:22:36.366 [main] TRACE i.g.b.seljup.ScreenshotManager - Creating screenshot for RemoteWebDriver: chrome on LINUX (2a505763be0a9438d1d8e71c82bd830d) in ./target/failsafe-reports/testRecordingMethod_arg0_chrome_2a505763be0a9438d1d8e71c82bd830d.png
19:22:36.375 [main] INFO  i.g.b.wdm.docker.DockerService - Stopping Docker container selenoid/video-recorder:7.1
19:22:36.376 [main] TRACE i.g.b.wdm.docker.DockerService - Stopping container 6cbda9b7328217be4ee724e26eb6a46c302eaf0f789e2c82e14528c4c27c55b0 (timeout 5 seconds)
19:22:37.292 [main] TRACE i.g.b.wdm.docker.DockerService - Removing container 6cbda9b7328217be4ee724e26eb6a46c302eaf0f789e2c82e14528c4c27c55b0
19:22:37.301 [main] TRACE i.g.b.seljup.SeleniumJupiter - Single session false
19:22:37.301 [main] TRACE i.g.b.seljup.SeleniumJupiter - Quitting contextId [engine:junit-jupiter]/[class:org.example.RecordingFailsafeOutputFolderTest]: (wdmMap={[engine:junit-jupiter]/[class:org.example.RecordingFailsafeOutputFolderTest]=[io.github.bonigarcia.wdm.managers.ChromeDriverManager@6c0d7c83]})
19:22:37.302 [main] DEBUG i.g.bonigarcia.wdm.WebDriverManager - Quitting RemoteWebDriver: chrome on LINUX (2a505763be0a9438d1d8e71c82bd830d)
19:22:37.359 [main] INFO  i.g.b.wdm.docker.DockerService - Stopping Docker container selenoid/vnc:chrome_97.0
19:22:37.360 [main] TRACE i.g.b.wdm.docker.DockerService - Stopping container 58136aed7a2c4b3aaaf70a1a75b946746a2f1e70279c8a3e2600bf18a976cffd (timeout 5 seconds)
19:22:37.620 [main] TRACE i.g.b.wdm.docker.DockerService - Removing container 58136aed7a2c4b3aaaf70a1a75b946746a2f1e70279c8a3e2600bf18a976cffd
19:22:37.635 [main] TRACE i.g.b.seljup.SeleniumJupiter - Removing managers from map (id [engine:junit-jupiter]/[class:org.example.RecordingFailsafeOutputFolderTest])
19:22:37.651 [main] TRACE i.g.b.seljup.SeleniumJupiter - Single session false

Error trace: none

[bonigarcia]

Maybe not by default, but I think that this could be interesting with a new configuration capability. Maybe something like this:

seleniumJupiter.getConfig().useQualifiedClassNameAsOutputFolder();

If this were enabled, the recordings and screenshots would be organized in new folders using the qualified name of the test classes. Is that what you want, right?

I think you quite well the Selenium-Jupiter code base. Are you up to create a PR about it? Thanks in advance.

[danielFesenmeyer]

Hi @bonigarcia, I would be happy to provide a PR - but it may take a month before I find the time to do it. Is that ok?

I like the idea with the additional config option. Just wondering what happens if useQualifiedClassNameAsOutputFolder is false and useSurefireOutputFolder is true? Maybe it's best to ignore useQualifiedClassNameAsOutputFolder in this case to ensure backwards compatibility?

[bonigarcia]

@danielFesenmeyer No problem, your PR will be welcome whenever you can. Regarding your question, maybe yes, it could make sense to ignore useQualifiedClassNameAsOutputFolder when both options are enabled.

[danielFesenmeyer]

Hi @bonigarcia,

It took some time, but finally I managed to provide the PR ;)

[bonigarcia]

Thanks a lot for contributing, @danielFesenmeyer!

[danielFesenmeyer]

You are welcome. Thanks for the quick merge :smiley: