CLI options for test execution

[coinzdude]

Consider a robust set of CLI options for test execution, the goal to allow for CI/CD and to capture robust test results

ERROR(S):
  Required option 's, side' is missing.
  Required option 'u, url' is missing.

  -v, --verbose                     set output to verbose

  -s, --side                        Required. source side file

  -n, --nameOfTest                  Name of specific test (instead of suite)

  -t, --testSuite                   test suite name

  -b, --browser                     browser, Chrome or Firefox

  -d, --dataFile                    JSON input file with variable values to use
                                    with the test

  -g, --generateDataTemplateFile    generate data input template file from test
                                    suite SIDE file into specified target JSON
                                    file name. All matches are used, testSuite
                                    is ignored. This option skips test
                                    execution.

  -i, --interactive                 use keyboard to control test steps, pausing
                                    at each step

  -u, --url                         Required. base url for test, e.g.
                                    https://site.com

  -p, --pause                       pause at script-name,line#

  -o, --outputFolder                output folder

  -e, --endOfTestScreenshots        get screenshot at end of each test

  -h, --haltOnJSError               Stop scripts on JavaScript error

  --help                            Display this help screen.

  --version                         Display version information.

[coinzdude]

Along with this, consider the generated test results document would be a reportable JSON structure, and that the test can capture per command test execution timings.

{
  "TestID": "dev.saas.-2022-04-17T21:30:25",
  "ScriptName": "CC",
  "TargetSite": "https://dev.saas./",
  "TestTimeUtc": "2022-04-17T21:30:25.449322Z",
  "Results": [
    {
      "Name": "case-tracker",
      "ExecutionTime": 648.29,
      "TestFailed": false,
      "ConsoleErrors": true,
      "LineIfFailed": null,
      "LastErrorMessage": null,
      "TestTimings": {
        "Name": "case-tracker",
        "Timings": [
          {
            "CommandLine": 1,
            "Time": 10.86,
            "Command": "run|login||"
          },
          {
            "CommandLine": 2,
            "Time": 0.0,
            "Command": "mouseOver|css=#sideNavItems > #case-tracker-navlink .fa-list-alt||"
          },
          {
            "CommandLine": 3,
            "Time": 0.78,
            "Command": "click|css=#sideNavItems > #case-tracker-navlink .fa-list-alt||"
          },
          {
            "CommandLine": 4,
            "Time": 0.14,
            "Command": "click|id=MeButton||"
          },
          {
            "CommandLine": 5,
            "Time": 0.12,
            "Command": "click|id=TeamButton||"
          },
          {
            "CommandLine": 6,
            "Time": 0.05,
            "Command": "click|id=FollowButton||"
          },
          {
            "CommandLine": 7,
            "Time": 0.01,
            "Command": "mouseOver|id=FollowButton||"
          },
          {
            "CommandLine": 8,
            "Time": 0.0,
            "Command": "mouseOut|id=FollowButton||"
          },
          {
            "CommandLine": 9,
            "Time": 0.0,
            "Command": "mouseOver|id=AllButton||"
          },
          {
            "CommandLine": 10,
            "Time": 0.05,
            "Command": "click|id=AllButton||"
          },
          {
            "CommandLine": 11,
            "Time": 0.0,
            "Command": "mouseOut|id=AllButton||"
          },
          {
            "CommandLine": 12,
            "Time": 622.91,
            "Command": "click|css=.col-sm-12:nth-child(2) .text-reset||"
          },
          {
            "CommandLine": 13,
            "Time": 0.07,
            "Command": "click|css=.col-sm-12:nth-child(2) .text-reset||"
          },
          {
            "CommandLine": 14,
            "Time": 0.06,
            "Command": "click|css=.col-sm-12:nth-child(2) .text-reset||"
          },
          {
            "CommandLine": 15,
            "Time": 0.03,
            "Command": "doubleClick|css=.col-sm-12:nth-child(2) .text-reset||"
          },
          {
            "CommandLine": 16,
            "Time": 0.11,
            "Command": "click|css=.col-lg-4:nth-child(2) .stretched-link||"
          },
          {
            "CommandLine": 17,
            "Time": 0.12,
            "Command": "click|css=.col-lg-4:nth-child(2) .stretched-link||"
          },
          {
            "CommandLine": 18,
            "Time": 2.24,
            "Command": "selectWindow|handle=${win4}||"
          },
          {
            "CommandLine": 19,
            "Time": 0.01,
            "Command": "mouseOver|css=#sideNavItems > #case-tracker-navlink .fa-list-alt||"
          },
          {
            "CommandLine": 20,
            "Time": 0.62,
            "Command": "click|css=#sideNavItems > #case-tracker-navlink .fa-list-alt||Test Validation"
          }
        ]
      }
    }
  ]
}

[toddtarsi]

Closing this. The side-runner is always getting better option applications and tuning, and jest can be configured for this sort of thing via the jestOptions flag.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.