Add the ability to return a Response that indicates that a Step has been skipped

[Chralex]

Basically a way to skip a step, meaning that it will not be included in the statistics, this is useful when doing a multi-step scenario where step behavior relies on the result of the previous step, or in cases where a step only has to be run once per scenario for example a login-step that authenticates the user, when we get back to that step we would like to skip it, since it is not realistic that the user would re-authenticate at this point.

Also what I mean by skip is that it should just proceed to the next Step in the Scenario and not stop the flow.

[AntyaDev]

Hi @Chralex. Regarding modeling authentication or what you called init step: for this you have Scenario.TestInit or ConnectionPool.

[AntyaDev]

Can you please describe a few real cases where you need Skip step.

[Chralex]

@AntyaDev I will come with some more examples, but I'd also like to say that we would want to include the authentication step in the statistics to see how long authentication takes, and over long time our users will have to re-authenticate because their access token expires.

[Chralex]

One example that I have is that we have load profiles where the different profiles has a certain percentage chance of performing an action that will put their user in a state, in that state they can perform some steps but not others. This could also be solved by making "Conditional Steps" where a condition is checked before the Step is run. Basically some APIs are only available for the users that contracted previous APIs in a certain way.

Here is some real code that I am using at the moment.

private async Task<Response> ClientQueryOffMapLobby(StepContext<LoadTestClient> context)
{
	LoadTestClient client = context.Connection;

	if (client.Character.IsOffMap)
	{
		if (!client.Character.LastOffMapLobbyQueryTime.HasValue || client.Character.LastOffMapLobbyQueryTime.Value.Add(OffMapLobbyQueryInterval) <= SimulationTime)
		{
			client.Character.LastOffMapLobbyQueryTime = SimulationTime;

			BackendApiOffMapLobbyQueryResponse offMapLobbyResponse =
				await context.Connection.SendMessage<BackendApiOffMapLobbyQueryRequest, BackendApiOffMapLobbyQueryResponse>(new BackendApiOffMapLobbyQueryRequest() { });

			if (offMapLobbyResponse != null)
				return Response.Ok();
			else
				return Response.Fail();

			//return Response.OkIfNotNull(offMapLobbyResponse);
		}
	}

	//return Response.Fail();
	return Response.Skip();
}

[AntyaDev]

I think conditional steps is a better fit here and simplify the flow since Step.Skip() is adding variety to every step which is actually not needed. On the other hand, it's not clear how to calculate latency for such steps, RPS. The condition could be defined as optional argument of Step.Create.

[Chralex]

@AntyaDev Alright, I will probably look into making a pull request that implements conditional steps instead, I think it would be appropriate to have two overloads, one that allows you to specify one condition and another one which is a params of conditions in case the developer feels like it is easier to read multiple smaller evaluations than one big one.

I will probably look into it once I've finished trying out the clustering that is provided, I need to do some distributed tests as soon as possible so it might take a while before I get back to this.

[AntyaDev]

@Chralex thank you for good input. Regarding clustering - in the next version it will be simplified, I am going to introduce MQTT protocol for communication plus agent groups will appear. So you basically don't need to specify IP address of every agent. I hope I will release a new version till Monday. Regarding PR, we can talk over skype to properly discuss your cases and implementation detail.

[AntyaDev]

Hi @Chralex Just want to note about a new cluster membership https://github.com/PragmaticFlow/NBomber/releases/tag/v0.14.0

[colinbobolin]

did this go anywhere? I see nothing about conditional steps, but it would be very helpful for us.

[AntyaDev]

Hi @colinbobolin ,

Could you please elaborate on the conditional steps? Do you have any examples?