testfx icon indicating copy to clipboard operation
testfx copied to clipboard

[Feature] Asserts supporting auto-retry/polling

Open Exoow opened this issue 1 year ago • 2 comments

Summary

Provide Assert methods that automatically retry the given function to assert until timeout.

Background and Motivation

Currently, my test code has asserts like these:

await Assert.That.ShouldBeEqual(value, GetValueAsync);
await Assert.That.ShouldBeTrue(async () => options.RowCount <= await GetRowCountAsync());

These are custom extensions that simply wrap around a Retry object that polls until success or a timeout.

public static async Task ShouldBeTrue(this Assert assert, Func<Task<bool>> condition, string? message = null, RetrierOptions? options = null)
{
    await retrier.Retry(async () => Assert.IsTrue(await condition(), message), options);
}

public static async Task ShouldBeEqual(this Assert assert, string expected, Func<Task<string?>> function, string? message = null, RetrierOptions? options = null)
{
    await retrier.Retry(async () => Assert.AreEqual(expected, await function(), message), options);
}

Proposed Feature

Ideally I can replace the code above by the following. Note, non-async counterparts are also useful.

await Assert.ToBeEqualAsync(T expected, Func<Task<T>> retryFunction, string message);
await Assert.ToBeTrueAsync(Func<Task<T>> retryFunction, string message);
// and this where the function should not throw an error
Assert.ToPass(Action retryFunction, string message);
await Assert.ToPassAsync(Func<Task> retryFunction, string message);

Exoow avatar Feb 01 '24 12:02 Exoow

Hi @Exoow,

Thanks for the suggestion!

I have to admit I am personally not fan of this feature request as I believe the assertion should only check values and that there are already many retry solutions available out there (e.g. https://github.com/App-vNext/Polly).

But I will keep this open for some time to collect thumbs up/down.

Evangelink avatar Feb 01 '24 14:02 Evangelink

Hello @Evangelink - I understand, considering retries are only applicable in test levels beyond unit tests.

The Polly library seems a bit overkill in terms of syntax/features for simple asserts. In case this doesn't make it, I'll stick to my own extension methods. Thanks for your feedback! :)

Exoow avatar Feb 01 '24 15:02 Exoow

@Exoow I will move forward by closing this issue as won't fix as there was no traction.

Thank you again for suggesting this feature.

Evangelink avatar Mar 17 '24 17:03 Evangelink