nightwatch icon indicating copy to clipboard operation
nightwatch copied to clipboard

HTML reporter should not be produced in case of type errors

Open mostmentor opened this issue 2 years ago • 5 comments

Description of the bug/issue

I was running set of tests (27 tests) and one of the tests was calling a variable from globals that doesn't exist, This test status never reported and wasn't able to notice that the test was being skipped

Also the test is never reported in HTML reporter and this is how it looks like image

Steps to reproduce

No response

Sample test

describe('sample test', function() {
  const resource = this.globals.trials.saas_only_trial; //Issue happened here, This variable doesn't exist in my global
     
//Some steps here

    it("End", function(browser) {
      browser.end();
    });

  });

Command to run

No response

Verbose Output

No response

Nightwatch Configuration

const settings = {
     "src_folders": [
          "tests/saas/C93573-accessSaaSProduct.js"
     ],
     "page_objects_path": [
          "./page_object"
     ],
     "globals_path": "globals.js",
     "webdriver": {
          "keep_alive": true
     },
     "custom_commands_path": [
          "./lib/custom_commands",
          "./node_modules/nightwatch-xhr/es5/commands"
     ],
     "custom_assertions_path": [
          "./node_modules/nightwatch-xhr/es5/assertions"
     ],
     "test_workers": {
          "enabled": false,
          "workers": 0
     },
     "test_settings": {
          "default": {
               "localOption": true,
               "videoRecorder": false,
               "skip_testcases_on_fail": false,
               "end_session_on_fail": false,
               "disable_error_log": false,
               "launch_url": "https://nightwatchjs.org",
               "screenshots": {
                    "enabled": false,
                    "path": "screens",
                    "on_failure": true
               },
               "desiredCapabilities": {
                    "browserName": "chrome"
               },
               "webdriver": {
                    "port": 9515,
                    "start_process": true,
                    "server_path": "node_modules/chromedriver/lib/chromedriver/chromedriver.exe"
               }
          }
     },
     "live_output": true,
     "detailed_output": true,
     "test_runner": {
          "type": "default",
          "options": {
               "bail": false,
               "retries": 0
          }
     }
}
                  module.exports = new Promise(resolve => {
                    resolve(settings);
                  });
                  module.exports.settings = settings

Nightwatch.js Version

2.6.7

Node Version

16.19.0

Browser

chrome

Operating System

win10

Additional Information

No response

mostmentor avatar Jan 09 '23 19:01 mostmentor

@mostmentor Thanks for reporting the issue. Can you please attach the HTML Report here? It would help us a lot in detecting the root cause of the problem.

vaibhavsingh97 avatar Jan 16 '23 09:01 vaibhavsingh97

index.zip

@vaibhavsingh97 Here is the HTML report

mostmentor avatar Jan 16 '23 12:01 mostmentor

Thanks, @mostmentor, for the HTML Report.

From the looks of it, Nightwatch didn't report anything since it's more of a javascript error, and the front end broke because of it. A nice error message should be the right fix.

vaibhavsingh97 avatar Jan 16 '23 13:01 vaibhavsingh97

Is it possible for nightwatch to fail the test because of javascript errors ? I will never know the test is not running if it's not reported in nightwatch as failure

mostmentor avatar Jan 16 '23 14:01 mostmentor

Is it possible for nightwatch to fail the test because of javascript errors ?

Nightwatch never ran the tests and failed when it required all the files to run it. Since there was a javascript issue, you must have seen the Actionable Error Message. This is an issue, as HTML Report should never be written in such a case, as the e2e test never ran.

Ref: image

vaibhavsingh97 avatar Jan 17 '23 08:01 vaibhavsingh97