Improve discovery & execution performance in interactive scenarios

[peterwald]

The purpose of this issue is to gather community feedback on some strategies for improving the latency of the test platform esp. when used in user-interactive scenarios (e.g. Visual Studio Test Explorer).

Some of the areas we've identified include:

1. Concurrent Test Discovery - In solutions that contain a large number of tests and test assemblies, can we run discovery on each assembly in parallel? Furthermore, can we parallelize the discovery of tests within each assembly?
2. Source-Based Partial Test Discovery - Using only the source code, we can discover the most common kinds of tests more quickly without requiring the test assemblies to be recompiled.
3. Incremental Test Discovery - By considering only recent changes in the source code and merging them with tests that have been previously discovered, we can more quickly keep discovery data up to date with the code.
4. Efficient TestCase Serialization - TestCase objects from the discovery process are serialized and sent back and forth between the test discovery/execution processes. Can we make the serialization more efficient or eliminate it entirely in some cases?

Comments and feedback are welcome

CC: @ManishJayaswal @jfleisher

[ManishJayaswal]

@codito FYI

[gulbanana]

concurrent test discovery would be fantastic for me. i have a solution with nine test projects of roughly similar size, and plenty of cores to work on them

[Evangelink]

I believe this will be partially addressed by #3349 and maybe also through #2139.

@nohwnd do we want to explore some of the other points or shall I link this to #3349?