ensure plugin: add additional parameters to shutdown the artillery when SLO checks where failed

[savvagen]

1. Add a way to stop artilley (global.artillery.shutdown()) injector when conditions or thresholds will be failed.
2. Add additional parameter to the conditions and thresholds with boolean type to made a signal to artillery ensure plugin Example:

config:
  ensure:
    thresholds:
      - "metric.name.one": value1
         shutdown: true|false 
    conditions:
      - expression: "metric.name.one <= value1 and metric.name.two > value2"
         shutdown: true|false

[hassy]

@savvagen is the idea here to stop a running test early as soon as the condition is exceeded?

[savvagen]

@savvagen is the idea here to stop a running test early as soon as the condition is exceeded?

@hassy Yes. But to be able to handle that, we need to trigger checks periodically (as an example - on each report)... I watched the ensure plugin code, do we have some more extensions like this:

global.artillery.ext(
      {
        ext: 'beforeExit',
        method: async (data) =>

What kind of extensions do we also have in artillery?

[hassy]

Great suggestion!

Plugins can subscribe to a stats event which is emitted every 10 seconds, e.g. here the Datadog integration sets up a handler for that event to send metrics to Datadog:

https://github.com/artilleryio/artillery-plugin-publish-metrics/blob/master/lib/datadog.js#L95

[savvagen]

@hassy Grate. Then I will try to make a pool-request and implement the new shutdown flag under the stats event