programmatic API custom command doesn't fail the test

[Terence625]

Description of the bug/issue

When I'm using nightwatch programmatic API to launch the browser to do the test, the failure in custom command won't fail the test

Steps to reproduce

run node test.js, from the output, nightwatch log the Error, but the test.js didn't stop and keeps going until the last command

Sample test

// error.js, custom command
module.exports = class ErrorCommand {
  async command() {
    throw new Error("caught an error");
  }
};

// test.js
const Nightwatch = require("nightwatch");

const client = Nightwatch.createClient({
  headless: true,
  output: true,
  silent: false, // set to false to enable verbose logging
  browserName: "chrome", // can be either: firefox, chrome, safari, or edge
  timeout: 1000,
  env: null,
  desiredCapabilities: {},
  globals: {},
  parallel: false,
  disable_colors: false,
});

async function run() {
  try {
    const browser = await client.launchBrowser();
    await browser.url("https://www.google.com");
    await browser.error();
    await browser.url("https://www.google.com");
    await browser.quit();
  } catch (err) {
    console.log(err);
  }
}

run();

Command to run

node test.js

Verbose Output

→ Running command: url ('https://www.google.com')
  ⠋ Loading url: https://www.google.com
   Request POST /session/03ee5da98d6cdfec8ef837619b8bee17/url  
  ⠸ Loading url: https://www.google.com
   Response 200 POST /session/03ee5da98d6cdfec8ef837619b8bee17/url (1103ms)
  ℹ Loaded url https://www.google.com in 1107ms
 → Completed command: url ('https://www.google.com') (1107ms)
 
 → Running command: error ()
  Error
   caught an error

    Error location:
    /Users/wenguang.tian/nightwatch-test/nightwatch/commands/error.js:
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
     1 | module.exports = class ErrorCommand {
     2 |   async command() {
     3 |     throw new Error("caught an error"); 
     4 |     try {
     5 |       await this.api.element("ad").setValue(searchTerm);
    –––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––

 → Completed command: error () (3ms)
 
 → Running command: url ('https://www.google.com')
  ⠋ Loading url: https://www.google.com
   Request POST /session/03ee5da98d6cdfec8ef837619b8bee17/url  
  ⠴ Loading url: https://www.google.com
   Response 200 POST /session/03ee5da98d6cdfec8ef837619b8bee17/url (477ms)
  ℹ Loaded url https://www.google.com in 479ms
 → Completed command: url ('https://www.google.com') (479ms)
 
 → Running command: quit ()
   Request DELETE /session/03ee5da98d6cdfec8ef837619b8bee17  

   Response 200 DELETE /session/03ee5da98d6cdfec8ef837619b8bee17 (55ms)
   { value: null }
 Wrote log file to: /Users/wenguang.tian/nightwatch-test/logs/_chromedriver.log
 → Completed command: quit () (58ms)
 ChromeDriver process closed.

Nightwatch Configuration

No response

Nightwatch.js Version

3.1.2

Node Version

19

Browser

No response

Operating System

No response

Additional Information

No response

[gravityvi]

Hey @Terence625, could you try using this instead:

const error = new Error('caught an error');
error.abortOnFailure = true;
throw error;

[Terence625]

@gravityvi still the same result, the error is logged but the test isn't failed

[Terence625]

Hi, @gravityvi , update on this issue, I just found out, nightwatch official commands like click sendKeys also won't fail the test in programmatic api

[aakanksha88]

Hi @gravityvi @beatfactor we are also facing the same issue, could you please suggest any resolution