mcp icon indicating copy to clipboard operation
mcp copied to clipboard

[BUG] Flaky unit tests with `InternalServerError`

Open vcolin7 opened this issue 2 months ago • 2 comments

Describe the bug

Every now and then a build will fail with a message like the following:

[xUnit.net 00:00:01.22]       Assert.Equal() Failure: Values differ
[xUnit.net 00:00:01.22]       Expected: BadRequest
[xUnit.net 00:00:01.22]       Actual:   InternalServerError
[xUnit.net 00:00:01.22]       Stack Trace:
[xUnit.net 00:00:01.22]         D:\a\_work\1\s\tools\Azure.Mcp.Tools.KeyVault\tests\Azure.Mcp.Tools.KeyVault.UnitTests\Certificate\CertificateImportCommandTests.cs(132,0): at Azure.Mcp.Tools.KeyVault.UnitTests.Certificate.CertificateImportCommandTests.ExecuteAsync_ValidatesRequiredArguments(String argLine, Boolean shouldPassValidation)
[xUnit.net 00:00:01.22]         --- End of stack trace from previous location ---

I've seen it a bunch of times over the last month or two and I'm not sure why the server would just fail. Maybe we're trying to make a call faster than it can start? That said, I thought we're using mocks for unit tests, so there's that.

Expected behavior

All unit tests should always pass if no changes have been made to them.

Actual behavior

Tests occasionally fail with the aforementioned InternalServerError message.

Reproduction Steps

I don't think you can reproduce this, it's occasional.

Environment

CI pipelines on x64 builds

vcolin7 avatar Oct 21 '25 23:10 vcolin7

When did it first appear? Help us narrow this down please.

joshfree avatar Oct 22 '25 01:10 joshfree

I don't know exactly when it started appearing. I know I've seen it at least a few times in the past few weeks (a month?). Here's one more instance of this. It happen again in a KV test, which kind of helps us narrow things down a bit. I'll look into it come next week.

vcolin7 avatar Oct 24 '25 17:10 vcolin7