testfx icon indicating copy to clipboard operation
testfx copied to clipboard

Use a better serialization/deserialization mechanism for data discovery

Open Evangelink opened this issue 3 years ago • 7 comments

Describe the bug

Today, MSTest uses the DataContractJsonSerializer serializer in order to perform the serialization and deserialization of the data during discovery and execution.

The problem with current solution is that it requires custom objects to be marked with DataContractAttribute/DataMemberAttribute which isn't what user expects to do or to be doing for the sake of test.

At the moment, the BinaryFormatter seems to be the best option but it's not ideal as this formatter has a lot of security concerns associated (see https://learn.microsoft.com/dotnet/api/system.runtime.serialization.formatters.binary.binaryformatter#remarks)

Additional context

Related issues:

  • MS Test 2.2.4 - 2.2.10: Tests fail because DynamicData gets only structs with default values (#1094)
  • wrong value received by test method with ITestDataSource and values of struct with property of nullable primitive type (#1022)
  • DynamicData test is not passing exception objects correctly (#1037)
  • Using an interface in dynamic data causes a de-/serialization exception (#908)
  • DateTime equality test fails after upgrading from 2.2.3 (#1019)
  • Using dynamic data source method fails to work (#1588)
  • [DataRow]: Original nested types are lost when nested deeper than one level (#2390)
  • #3259
  • #1054
  • #3962

Evangelink avatar Dec 14 '22 09:12 Evangelink

Is there really any issue with using BinaryFormatter here? The data doesn't cross a security boundary, does it?

poizan42 avatar May 25 '23 11:05 poizan42

Technically I don't see much issues but we would be flagged by internal MS compliance checks + there are many discussions toward removing this formatter in some new version of .NET (nothing acted) so I am not sure if best is to be using that and wait for some "break" to change mechanism.

Evangelink avatar May 25 '23 12:05 Evangelink

Given BinaryFormatter Obsoletion Strategy it doesn't seem that using BinaryFormatter would be a good path forward.

jnyrup avatar May 29 '23 12:05 jnyrup

How about switching to System.Text.Json based serialization?

mungojam avatar Jul 03 '23 13:07 mungojam

I would need to make some tests to see how many issues this would fix. It might also be problematic for .NET fx users.

Having some more experience with mstest, I think that there is no silver bullet solution and I would probably need to check if the data can be serialized (or to be exact if serializing/deserializing data gives me the same data as input) in which case we can "expand" the data otherwise we should not expand and so don't try to serialize/deserialize data.

Evangelink avatar Jul 03 '23 14:07 Evangelink

It might also be problematic for .NET fx users.

To answer that specific thought, it's supported in .net standard 2.0 and v4.6.2

https://www.nuget.org/packages/System.Text.Json/

image

mungojam avatar Jul 03 '23 15:07 mungojam