Interaction between `--bail` & `--iteration-count`

[vrinek]

Assume collection.json with 2 requests and the following command:

newman run collection.json --bail --iteration-count 2

The behaviour I can observe now is that if the first request on the first iteration fails, nothing else is run.

The behaviour I would expect is if the first request on the first iteration fails, the second iteration still runs. The reason I expect this behaviour is that I use --iteration-count to test for flaky vs broken APIs and I use --bail because the calls in each collection are sequential (each one depends on globals set by the previous one).

Is the behaviour I describe above something that is possible with some options? Is it a potentially interesting addition to newman?

[kunagpal]

Hey @vrinek, you've made an interesting point there. @czardoz @shamasis

[shamasis]

I'm thinking of a concept of "bail mode" that can be passed optionally as parameter to --bail

[kunagpal]

@vrinek This is not a part of our current roadmap, but has been marked as a feature request and added to our internal tracker. We'll update this thread when any progress occurs on this. :)

[vrinek]

@kunagpal thanks for the update

[shamasis]

@czardoz can we make --bail also accept an optional parameter and then allow bailing modes? Say, --bail iteration. It could take iteration or collection and collection is set as default if nothing is set.