Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder has flaky test assertion

[ericstj]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=1056990 Build error leg or test failing: dotnet-watch.Tests.dll.3.WorkItemExecution Pull request: https://github.com/dotnet/sdk/pull/49092

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "Failed Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Sample failing log: https://helixr1107v0xdeko0k025g8.blob.core.windows.net/dotnet-sdk-refs-pull-49092-merge-89284afdacb94b978c/dotnet-watch.Tests.dll.3/3/console.3de535af.log?helixlogtype=result

  Failed Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder(usePolling: True) [1 s]
  Error Message:
   Expected:
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo1, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo2, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo3, Kind = Delete }
Actual:
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir, Kind = Update },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo1, Kind = Delete },
ChangedPath { Path = /tmp/dotnetSdkTests.igDIQiVU/True/subdir/foo2, Kind = Delete }

  Stack Trace:
     at Microsoft.DotNet.Watch.UnitTests.AssertEx.Fail(String message) in /_/test/dotnet-watch.Tests/Utilities/AssertEx.cs:line 214
   at Microsoft.DotNet.Watch.UnitTests.AssertEx.SequenceEqual[T](IEnumerable`1 expected, IEnumerable`1 actual, IEqualityComparer`1 comparer, String message, String itemSeparator, Func`2 itemInspector) in /_/test/dotnet-watch.Tests/Utilities/AssertEx.cs:line 169
   at Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.TestOperation(String dir, ChangedPath[] expectedChanges, Boolean usePolling, Action operation) in /_/test/dotnet-watch.Tests/FileWatcherTests.cs:line 63
   at Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder(Boolean usePolling) in /_/test/dotnet-watch.Tests/FileWatcherTests.cs:line 353
--- End of stack trace from previous location ---
  Standard Output Messages:

Have a look at the assertion the test is making -- https://github.com/dotnet/sdk/blob/0934942c262f12b26007f1cd3bad148d360becf5/test/dotnet-watch.Tests/FileWatcherTests.cs#L353-L390

It's asserting it will see a series of events exactly on different operating systems. FileSystemWatcher (and file notification APIs in the platform for that matter) don't garuntee that. You can't count on seeing all changed/updated events. Especially in the case of polling you cannot garuntee you'll see an update event if it's followed by a delete since a polling implementation can only see a file or directory has been removed, and not see any changes that might have happened between the last poll and removal.

This test should be changed to just assert on seeing a minimum set of events, not exact set. In other words - a contains check rather than equals.

Known issue validation

Build: :mag_right: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1056990 Error message validated: [Failed Microsoft.DotNet.Watch.UnitTests.FileWatcherTests.DeleteSubfolder] Result validation: :white_check_mark: Known issue matched with the provided build. Validation performed at: 6/2/2025 8:08:17 PM UTC

Report

Build	Definition	Test	Pull Request
1205092	dotnet/sdk	dotnet-watch.Tests.dll.3.WorkItemExecution	dotnet/sdk#51698

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	1