Add gracePeriod or timeout to error steps

[gpaul]

What would you like to be added:

Add a gracePeriod or timeout to the error teststep type.

Why is this needed:

I have a controller that migrates one type of object to another. I want to be able to specify that after some time, my original object no longer exists. Currently I cannot do that as it may take some amount of time for the controller to pick up the original object, create the derived object, and remove the original. Currently, the error teststep would fail immediately as the original object will likely still exist.

[kensipe]

@gpaul thinking through this feature req and your use case.

This is a good feature request... but I believe you could work around this missing feature with the following multi-step approach.

1. step 1, initiate whatever the controller looks for
2. step 1 assert, test (with timeout) for the output from the controller
3. step 2 assert in an errors.yaml file the thing that should be absent

[gpaul]

Unfortunately not - I thought about that too but the problem is that there is a race condition.

Once my object B is created from my object A, there is a moment in time when both A and B exist, but the controller has not yet removed object A. If I were to use an error.yaml to assert that object A no longer exists, then the moment after B gets created, the test would proceed to the 02-error.yaml and would fail because the controller hasn't removed object A yet.

1. Object A exists
2. Controller detects object A exists and creates object B
3. Object B exists yay! 01-assert.yaml succeeds
4. 02-error.yaml is executed and fails because object A still exists.
5. Controller deletes object A. ... I wish step 02-error.yaml had a grace period to allow the controller a few seconds to delete object A before immediately erroring due to the presence of object A.