selenium-ide
selenium-ide copied to clipboard
SeleniumHQ
selenium-side-driver 4.0.0-alpha.15 with '--output-format' option fails with error 'unknown option'
🐛 Bug Report
selenium-side-driver 4.0.0-alpha.15 with '--output-format' option fails with error 'unknown option'
To Reproduce
Steps to reproduce the behavior: % selenium-side-runner --filter="happy" --output-directory="side_output" --output-format="junit" demo_project.side error: unknown option '--output-format=junit'
Note that the usage info of v4 doesn't contain the usage of '--output-format' while v3 does: BTW, typo in 'maximimum' in the description of '-t, --timeout [number]'
Usage: selenium-side-runner [options] your-project-glob-here-.side [variadic-project-globs-.side]
Options: -V, --version output the version number --base-url [url] Override the base URL that was set in the IDE -c, --capabilities [list] Webdriver capabilities -s, --server [url] Webdriver remote server -p, --params [list] General parameters -f, --filter [string] Run suites matching name, takes a regex without slashes, eg (^(hello|goodbye).*$) -w, --max-workers [number] Maximum amount of workers that will run yourtests, defaults to number of cores (default: 8) -t, --timeout [number] The maximimum amount of time, in milliseconds, to spend attempting to locate an element. (default: 15000) -x, --proxy-type [type] Type of proxy to use (one of: direct, manual, pac, socks, system) -y, --proxy-options [list] Proxy options to pass, for use with manual, pac and socks proxies -n, --config-file [filepath] Use specified YAML file for configuration. (default: .side.yml) -o, --output-directory [directory] Write test results to files, format is defined by --output-format -f, --force Forcibly run the project, regardless of project's version -d, --debug Print debug logs -h, --help display help for command
And below is the usage info in v3:
Usage: selenium-side-runner [options] project.side [project.side] [*.side]
Options: -V, --version output the version number -c, --capabilities [list] Webdriver capabilities -s, --server [url] Webdriver remote server -p, --params [list] General parameters -f, --filter [string] Run suites matching name -w, --max-workers [number] Maximum amount of workers that will run your tests, defaults to number of cores --base-url [url] Override the base URL that was set in the IDE --timeout [number | undefined] The maximimum amount of time, in milliseconds, to spend attempting to locate an element. (default: 15000) --proxy-type [type] Type of proxy to use (one of: direct, manual, pac, socks, system) --proxy-options [list] Proxy options to pass, for use with manual, pac and socks proxies --config, --config-file [filepath] Use specified YAML file for configuration. (default: .side.yml) --output-directory [directory] Write test results to files, format is defined by --output-format --output-format [jest | junit] Format for the output. (default: jest) --force Forcibly run the project, regardless of project's version --debug Print debug logs -h, --help output usage information
Expected behavior
selenium-side-runner command with '--output-format' option should be successful.
Project file reproducing this issue (highly encouraged)
Any valid .side file can be used to repro this issue.
Environment
OS: MacOS Monterey 12.6 Selenium IDE Version: v3.17.0 Selenium SIDE Runner Version: 4.0.0-alpha.15 Node version: v18.10.0 ChromeDriver version: 106.0.5249.61 Browser: Browser Version: 106.0.5249.61
+1 this param is described in the official docs below and it leads the user to an error that it would be already available to be used https://www.selenium.dev/selenium-ide/docs/en/introduction/command-line-runner#output-test-results-to-a-file
+1 Same here cannot get any output report "output-directory=results" "--output-format=junit" >> option unknown
@toddtarsi OK, but how do you get any test output results then?
Doing this doesn't give anything in the results folder:-
selenium-side-runner -o results Simple.side
I just use -d to get any debug info if I need it, and run tests primarily via a CI system, where the most important thing is exit codes and failure points. If there was anything I'd really want to improve, it would be the stack traces on failure.
@jezahl - If you're really looking to do stuff, I left an escape hatch to pass in whatever jest CLI options you need (I use --forceExit personally because the browser vendor tunnels are a nightmare from jest and Github actions):
selenium-side-runner --jest-options '"--outputFile=./your-file.txt"' ./your-test.side
Shorthand version:
selenium-side-runner -j '"--outputFile=./your-file.txt"' ./your-test.side
@balimoon777 you can (with [email protected] or later) generate a junit report after installing jest-junit (npm install -g jest-junit) and configuring it.
JEST_JUNIT_OUTPUT_DIR=side_output JEST_JUNIT_OUTPUT_NAME=selenium-side-report.xml selenium-side-runner -j " --reporters=jest-junit --reporters=default " --filter="happy" demo_project.side
@Baltazardoung - You, sir or madam, are the real mvp. I can't praise you highly enough.
I want to close this, but honest to god I don't want to lose your note @Baltazardoung
@balimoon777 you can (with
[email protected]or later) generate ajunitreport after installing jest-junit (npm install -g jest-junit) and configuring it.JEST_JUNIT_OUTPUT_DIR=side_output JEST_JUNIT_OUTPUT_NAME=selenium-side-report.xml selenium-side-runner -j " --reporters=jest-junit --reporters=default " --filter="happy" demo_project.side
How do i do this ? "JEST_JUNIT_OUTPUT_NAME" is that a path variable ? I'm on Windows
@chrisisth - That is an environment variable (https://docs.oracle.com/en/database/oracle/machine-learning/oml4r/1.5.1/oread/creating-and-modifying-environment-variables-on-windows.html)
How about removing the option from the docs if there is no plan to implement it?
@miiihi - I completely agree. Would you point me to the docs where it's at?
Here https://www.selenium.dev/selenium-ide/docs/en/introduction/command-line-runner#output-test-results-to-a-file