Unit tests for linters sometimes seem to disregard PPDs

[3geek14]

I've noticed two issues with unit tests I've attempted to write, both of which involve PPDs. I'm willing to believe I'm at fault, but I think there's a bug.

First:

I inherited a custom analyzer that checked if a line was longer than our column limit. This analyzer didn't respect #pragma warning directives, so it was impossible to disable the analyzer for intentionally-too-long lines. After fixing this, I wanted to add a unit test to this analyzer (and our other analyzers) to verify that they respect pragma directives. Here are the two tests I wrote:

  [Test]
  public async Task IgnoreAfterPragmaDisable() {
    const string testCode = @"
using System;
public class Foo {
  #pragma warning disable CA0000
  public static void ThisIsAVeryLongFunctionNameBecauseIWantToMakeThisLineFarFarTooLong() => Console.WriteLine();
  #pragma warning restore CA0000
}
";
    await Verifier.VerifyAnalyzerAsync(testCode);
  }

  [Test]
  public async Task ReportAfterPragmaRestore() {
    const string testCode = @"
using System;
public class Foo {
  #pragma warning disable CA0000
  #pragma warning restore CA0000
{|#0:  public static void ThisIsAVeryLongFunctionNameBecauseIWantToMakeThisLineFarFarTooLong() => Console.WriteLine();|}
}
";
    DiagnosticResult expected =
        Verifier.Diagnostic("CA0000").WithLocation(0);
    await Verifier.VerifyAnalyzerAsync(testCode, expected);
  }

The first test passes, but the second fails. Here's the error message for the second one:

  Context: Verifying exclusions in '#pragma warning disable' code
Mismatch between number of diagnostics returned, expected "0" actual "1"

Diagnostics:
// /0/Test0.cs(7,1): warning CA0000: Lines must be no more than 100 characters in length
VerifyCS.Diagnostic().WithSpan(7, 1, 7, 114),


  Expected: 0
  But was:  1
   at Microsoft.CodeAnalysis.Testing.Verifiers.NUnitVerifier.Equal[T](T expected, T actual, String message) in
[stack trace omitted]

Interestingly, if I remove , expected from the final line of test, leaving the test code unchanged, I get a different error message:

  Mismatch between number of diagnostics returned, expected "0" actual "1"

Diagnostics:
// /0/Test0.cs(6,1): warning CA0000: Lines must be no more than 100 characters in length
VerifyCS.Diagnostic().WithSpan(6, 1, 6, 114),


  Expected: 0
  But was:  1
   at Microsoft.CodeAnalysis.Testing.Verifiers.NUnitVerifier.Equal[T](T expected, T actual, String message) in
[stack trace omitted]

Somehow, expecting a diagnostic causes the actual diagnostic to change.

Second:

I'm trying to write an analyzer to enforce our indentation rules. There's a bug in my code involving PPDs near the ends of files. Here's a file that my analyzer flags as wrong:

public class Foo {
  #region Bar
  public int Baz;
  #endregion
}

My analyzer reports a diagnostic at the EndOfFileToken. I wanted to debug this issue, so I tossed this small example into a unit test:

  [Test]
  public async Task PpdAtEof() {
    const string testCode = @"
public class Foo {
  #region Bar
  public int Baz;
  #endregion
}
";
    await Verifier.VerifyAnalyzerAsync(testCode);
  }

I expect this test to fail, because of the bug in my analyzer. However, it passes.

Conclusion:

I've encountered multiple issues when writing unit tests that arise when I'm trying to test how my analyzers behave around PPDs. I don't have a lot of reason to expect them to be related, except that they both involve PPDs.