dotnet
[API Proposal]: Extend FakeLogCollector with waiting capabilities
Latest proposal version
Background and motivation
Sometimes you don't know the exact moment in testing when a log will be recorded, particularly if it's part of a background job or something similar, or for convenience you may use it to assert if something has finished before carrying out further assertions.
API Proposal
public class FakeLogCollector
{
/*
... existing APIs ...
public void Clear()
public IReadOnlyList<FakeLogRecord> GetSnapshot(bool clear = false)
public FakeLogRecord LatestRecord
public int Count
*/
public IAsyncEnumerable<FakeLogRecord> GetLogsAsync(CancellationToken cancellationToken = default)
}
Behavior and usage
- Each enumerator always starts the iteration from the beginning of the currently stored (in memory) logs.
- In order to await a state of logs that come strictly after a certain known state and to not miss any logs coming in between the known state and the newly started wait, the caller needs to track an index of the previously known state, make sure not to clear the logs and skip the known count of logs during the new iteration. The asserting/waiting condition is then applied only to the logs coming after the previously known state.
- Using a shared enumerator across these waiting instances is also an option and not requiring to track indexes, re-iterating and skipping, but the usage of timeout-based cancellation logic becomes problematic, especially for net462 which does not support token reset.
With a library such as System.Linq.Async the caller can also do for example the following, which is expected to be among the more common scenarios:
var myLogs = await fakeLogCollector.GetLogsAsync(ct).Where(log => log.Category == "x").Take(3).ToListAsync();
Alternative designs
With optional start index
public class FakeLogCollector
{
/* ... existing code ... */
public IAsyncEnumerable<FakeLogRecord> GetLogsAsync(
int? startIndex = null, // ⬅️
CancellationToken cancellationToken = default
)
}
This API would allow for a more optimal solution when needing to start waiting from a previously known log state (assuming no Clear() is called). The caller would need to keep track of the previously awaited state, but would not need to perform re-iteration of log records that are not interesting.
With int count argument
public class FakeLogCollector
{
/* ... existing code ... */
public IAsyncEnumerable<FakeLogRecord> GetLogsAsync(
int? count = null, // ⬅️
CancellationToken cancellationToken = default
)
}
The iteration would stop when the number of returned logs reaches the count. However, this can be achieved by Skip(c) or custom iteration skipping as well. Also, the scenario of awaiting a fixed amount of log does not seem to be very useful here as opposed to MetricCollector where a minCount is awaited in order to retrieve a measurement snapshot.
Original proposal (archived)
Background and motivation
Sometimes you don't know the exact moment in testing when a log will be recorded, particularly if it's part of a background job or something similar, or for convenience you may use it to assert if something has finished before carrying out further assertions.
API Proposal
Basically this would be tacked on to the FakeLogCollector. The api is very similar to what the MetricCollector does. By default the timeout would be 5 seconds. I think typically the default would be to wait indefinitely, although if I'm writing a test I would find that annoying when a test hangs indefinitely if it fails. Maybe adding an option to set the default is another option.
Typically I'm waiting for certain logs in particular, so the most common thing for me would be to filter based on category.
public class FakeLogCollector
{
public ValueTask<FakeLogRecord?> WaitForLogAsync(CancellationToken cancellationToken);
public ValueTask<FakeLogRecord?> WaitForLogAsync(int millisecondsTimeout, CancellationToken cancellationToken);
public ValueTask<FakeLogRecord?> WaitForLogAsync(TimeSpan timeout, CancellationToken cancellationToken);
public ValueTask<FakeLogRecord?> WaitForLogAsync(int millisecondsTimeout);
public ValueTask<FakeLogRecord?> WaitForLogAsync(TimeSpan timeout);
public ValueTask<FakeLogRecord?> WaitForLogAsync(Func<FakeLogRecord, bool> filter, CancellationToken cancellationToken);
public ValueTask<FakeLogRecord?> WaitForLogAsync(Func<FakeLogRecord, bool> filter, int millisecondsTimeout, CancellationToken cancellationToken);
public ValueTask<FakeLogRecord?> WaitForLogAsync(Func<FakeLogRecord, bool> filter, TimeSpan timeout, CancellationToken cancellationToken);
public ValueTask<FakeLogRecord?> WaitForLogAsync(Func<FakeLogRecord, bool> filter, int millisecondsTimeout);
public ValueTask<FakeLogRecord?> WaitForLogAsync(Func<FakeLogRecord, bool> filter, TimeSpan timeout);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(Func<FakeLogRecord, bool> filter, int minCount, CancellationToken cancellationToken);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(Func<FakeLogRecord, bool> filter, int minCount, int millisecondsTimeout, CancellationToken cancellationToken);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(Func<FakeLogRecord, bool> filter, int minCount, TimeSpan timeout, CancellationToken cancellationToken);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(Func<FakeLogRecord, bool> filter, int minCount, int millisecondsTimeout);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(Func<FakeLogRecord, bool> filter, int minCount, TimeSpan timeout);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(int minCount, CancellationToken cancellationToken);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(int minCount, int millisecondsTimeout, CancellationToken cancellationToken);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(int minCount, TimeSpan timeout, CancellationToken cancellationToken);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(int minCount, int millisecondsTimeout);
public ValueTask<IReadOnlyList<FakeLogRecord>> WaitForLogsAsync(int minCount, TimeSpan timeout);
}
API Usage
An example waiting for a single log entry.
// At some point the fake log collector is made
var collector = new FakeLogCollector();
// imagine some logs are being written asynchronously
// if the log has already been recorded, returns immediately, otherwise will wait a matching log is found or until a certain amount of time is reached
var record = await collector.WaitForLogAsync(r => r.Category == "MyCategory");
// user can then do their own asserts on the results
Assert.That(record, Is.Not.Null);
Assert.That(record, Has.Message.EqualTo("my message"));
An example waiting for multiple log entry.
// At some point the fake log collector is made
var collector = new FakeLogCollector();
// imagine some logs are being written asynchronously
// if enough logs have already been recorded, returns immediately with logs matching the condition, otherwise will wait a certain number of matching log are found or until a certain amount of time is reached
var multipleRecords = await collector.WaitForLogsAsync(r => r.Category == "MyCategory", 2);
// user can then do their own asserts on the results
Assert.That(multipleRecords, Has.Count.EqualTo(2));
Assert.That(record[0], Has.Message.EqualTo("my message"));
Alternative Designs
As mentioned above MetricCollector already does this to some extent. It separates the waiting and the retrieval of the records. How this would work with the added filtering which isn't present in MetricCollector I'm not quite sure. I do find for convenience it's handy to do the filtering an retrieving all in one method.
Risks
It will likely indirectly impact parts of the underlying api that is already in use due to changes in how logs are collected so that they can be awaited on.
