Create analyzer for non-serializable TheoryData type arguments (xunit/xunit#2866)

[jared-chevalier]

Suggested initial version of a solution for issue xunit/xunit#2866.

Includes analyzer for proposed new rule xUnit1044: TheoryData type arguments should be serializable.

Please let me know if you have any feedback.

Solution

Serializability enumeration

• NeverSerializable
• PossiblySerializable
• AlwaysSerializable

TheoryDataTypeArgumentFinder class

• Finds all TheoryData<> type arguments for a data source referenced by a given [ClassData] or [MemberData] attribute in a given test class, if applicable
• If the data source's type is not compatible with a TheoryData<> class (such as if it is a member with the type IEnumerable<object[]>), then no type arguments will be found
• If the data source's type derives from a TheoryData<> base class, then the type arguments for the base class will be found

SerializabilityAnalyzer class

• Analyzes a given type to determine whether it is always, possibly, or never serializable
• The static analysis of a type's serializability closely corresponds to the logic found in SerializationHelper and MemberDataAttributeBase from the xunit repository
• See discussion in issue xunit/xunit#2866

TypeSymbols class

• Lazy caching wrapper around TypeSymbolFactory (plus the Compilation and XunitContext), for reusing already generated type symbols

TheoryDataTypeArgumentsShouldBeSerializable analyzer

• Analyzes MethodDeclarationSyntax nodes with a [Theory] attribute
• For each [ClassData] or [MemberData] attribute: Uses TheoryDataTypeArgumentFinder to find all TheoryData<> type arguments for the referenced class or member, if applicable
• For each type argument found: Uses SerializabilityAnalyzer to check whether the type is serializable (unless the type should be ignored)
• For each never serializable type argument: Reports an xUnit1044 diagnostic for the data attribute syntax node, with a strongly phrased message: The type argument {0} is not serializable. Consider using a type that is known to be serializable.
• For each possibly serializable type argument: Reports an xUnit1044 diagnostic for the data attribute syntax node, with a weakly phrased message: The type argument {0} might not be serializable. Consider using a type that is known to be serializable.

Key points

Diagnostic severity

• The diagnostic currently has Info severity. Would Warning be more appropriate?
• Currently, using Warning would break the build because an existing test class, SetEqualityAnalyzerTests, uses a tuple type argument for a MatrixTheoryData data source, and warnings are treated as errors
• A solution would be to skip analyzing data sources associated with DisableDiscoveryEnumeration = true, but that has not been implemented yet (see discussion in issue xunit/xunit#2866)

Statically analyzing serializability of enumerations

• See discussion in issue xunit/xunit#2866
• Any given enumeration type is either always serializable (from a local assembly) or never serializable (from the GAC)
• As far as I can tell, static analysis cannot determine whether an enumeration type is from a local assembly or from the GAC
• As a result, all enumeration types are treated as possibly serializable
• However, the analyzer ignores them, in order to prevent a diagnostic from being found for every single enumeration type

IXunitSerializable type symbol

• IXunitSerializable is in namespace Xunit.Abstractions in xUnit version 2, but in Xunit.Sdk in version 3
• The version 2 type symbol is accessible via a lazy cache in xunitContext.V2Abstractions?.IXunitSerializableType (as well as directly via TypeSymbolFactory)
• There does not seem to be any corresponding property in any of the version 3 contexts. Would it make sense to add one somewhere in the contexts? Ideally there would be a common interface, implemented for each version, that provided the type symbol for the current version
• I added the TypeSymbolFactory.IXunitSerializable_V3 factory method for the version 3 type symbol
• My TypeSymbols class currently uses the following logic to access the appropriate type symbol: TypeSymbolFactory.IXunitSerializable_V3(compilation) ?? xunitContext.V2Abstractions?.IXunitSerializableType