[Feature Request] Pre-test script runner

[colceagus]

Hi,

Following #73 , to generate random data for test input, I wrote a script to generate a csv with random data for my tests.

I want to include an option in the artillery test, for a pre-script runner, with validation option (error, success).

I want to run my script to generate new data before running the test, so I can provide the/a new payload (with new generated content) file to the test.

Is there a way to run a nodejs file before running the test with artillery ?

Regards, Daniel.

[hassy]

Not yet. Any reason you wouldn't run the Node.js script before running Artillery? (in your shell script / Makefile)

[mrchief]

Any reason you wouldn't run the Node.js script before running Artillery?

Its not cohesive and self contained then. Another scenario I have is to be able to login (once) and obtain some credentials and then use those for rest of the test.

You can almost do anything with shell scripting, but then, with that argument, almost everything is redundant, isn't it? :)

[hassy]

@mrchief Logging in should probably be part of the scenario itself in your use case. Running external scripts to generate payload data is the perfect use case for something like make - Artillery is not a task runner.

[mrchief]

Is there a way to say that run login only once under scenario? What if I'm not trying to benchmark login + API workflow, just the API?

[hassy]

@mrchief If you have long-lived session tokens or something like that which you want to use in test scenarios rather than making auth calls inside the scenarios, then you'd need to pre-generate them. I'm open to including a pre-run hook similar to beforeRequest / afterResponse, but it wouldn't be a priority for the next release. PRs would be welcome though.

[mrchief]

I think a pre-run hook is all we're looking for here. I can't promise but will see if I can make a PR.

[colceagus]

@mrchief any updates on this one :D ?

[JohnieLee]

I have a use case for this. I'd like to load some context variables prior to running the scenarios. Right now, I make a call to a dummy endpoint and use beforeRequest to call a init script.

          "get": {
            "#description": "Dummy request to populate context variables",
            "url": "/hello",
            "beforeRequest": "initContext"
          }

initContext:

    initContext(requestParams, context, ee, next) {
        const environment = TARGET_TO_ENVIRONMENT[context.vars.target];
        if (environment) {
            context.vars.config = require(`./configs/env.config.${environment}.json`);
        }
        return next();
    }

So, there's two different ways to solve this:

1. Add a pre-scenario hook or artillery init hook

"config": {
    "environments" : {
      "local" : {
        "target": "http://testEndpoint"
      },
    },
    "processor": "./helper.js",
    "init": "initContext"
  },

2. Add a context variable field:

"config": {
    "environments" : {
      "local" : {
        "target": "http://testEndpoingi"
      },
    },
    "context": "configs/env.config.json"
  },

[cskru]

I have a use case wherein I need to run a function that makes the server ready to benchmark and load test an api. This function has to run just once and before the scenarios are launched. An init config hook is much needed. Currently I do this manually. Run the function -> Take the necessary values after it's run -> Substitute it in the flow -> Run load test.

It'd be nice to have an init kinda hook.

[markdrury]

I'm a little surprised more people aren't squawking for this feature -- I'm facing the issue from @hassy's comment of almost two years ago wherein I need to generate a long-lived session token once before my load test begins, then use that token in all subsequent requests. I guess I'll need to use make or a shell script to wrap these discrete actions but that seems unnecessarily cumbersome. Am I missing something?

[mbruning24]

I have a use case as well. We're load testing a system where we want to create a particular record and then load test concurrent access + updates on that individual record. The creation would itself be a call that could be executed in a scenario. I would love to execute that as a flow step in some kind of phase init (or at least a custom function in the processor script file

config:
  init:
    - post:
        url: "/some/url"
        json: ...

OR

config:
  processor: "./my-fns.js"
  init: someInitFn

and in my-fns.js

module.exports = {
  someInitFn: (...args) => { ... }
}

[matkruse]

My case is the same as @markdrury as we should be able to perform an OAuth in the init and then only test any targeted endpoints of interest, at least initially. Stress testing as a whole with the OAuth is part of a different set of priorities.

[ryanwwest]

I have a similar use case where our containerized services use dynamic ports and I have to retrieve correct port (via some Javascript service calls) before doing load tests. I've gotten around this by using loops for now, but this would be a much nicer alternative.

[colceagus]

almost 4 years later... :)) might as well think this feature is really not that wanted. it's damn useful, tho.

[kwhitejr]

Keep the dream alive.

[endosama]

Same issue here... We need to make an initial network request to retrieve the list of keys we need to test with Artillery. Also could be useful to have a hook after all the scenarios have been completed, to add some logic around the metrics generated inside the JSON report.

[alexanderankin]

want to authenticate and set my header at the beginning with a hook. instead i have to hobble together some command line scripts to do this and put them in the environment before running artillery like its '99, and have headers property on every request. surely there has to be a better way.

[alexanderankin]

Logging in should probably be part of the scenario itself in your use case. Running external scripts to generate payload data is the perfect use case for something like make - Artillery is not a task runner.

Meanwhile in the README.md: (bold mine)

Artillery has a strong focus on developer happiness & ease of use, and a batteries-included philosophy.

[vegerot]

Any update on this?

[colceagus]

[henzlmarek]

It is possible to use beforeScenario hook and override $processEnvironment

env.yml

dev:
  PASS: 1234
prod:
  PASS: 4321

hooks.js

const yenv = require('yenv');
module.exports = {
    loadEnv: loadEnv,
}
/** Use this to load yenv
 * **/
function loadEnv(context, next) {
    context.vars.$processEnvironment = yenv('env.yml', {env: process.env.NODE_ENV});
    return next(); 
}

scenario.yml

config:
  plugins:
    expect: {}
  target: 'https://'
  phases:
      arrivalRate: 1
      duration: 1
  processor: '../helpers/hooks.js'
scenarios:
    beforeScenario:
      "loadEnv"
    flow:
      - log: "{{ $processEnvironment.PASS }}"

cross-env NODE_ENV=prod artillery run -k scenario.yml

log output: Started phase 0 (PHASE 1: Warm up the application), duration: 1s @ 19:33:28(+0100) 2021-02-02 / 1234

cross-env: multi-os posix support https://www.npmjs.com/package/cross-env

yenv: multi-env definition within yaml file https://www.npmjs.com/package/yenv

don't forget to add env.yml to gitignore :)

[GP-QA]

I also would like to have this feature. I need to login via the browser because of 0auth 2.0 authentication, grab the access token and re-use it in HTTP scenarios.