Use xunit2 for test discovery in-editor

[citizenmatt]

Currently, the in-editor test discovery uses the xunit1 API, rather than the xunit2 API that allows for custom test discovery. To use the xunit2 API to discover tests:

• [ ] Host an AppDomain that will be used for test discovery
• [ ] The custom AppDomain to (shadow) load the xunit2 API from the output directory. What happens if the project has been cleaned?
• [ ] Monitor the output directory for changes to any of the referenced assemblies (any of them might contain custom test discoverers), and tear down and reinitialise the AppDomain. This will have nasty implications for performance - reinitialising the AppDomain on each build. Can we get notified of the assemblies used for discovery, so we can only reset if the appropriate assemblies have changed?
• [ ] Once the tests are discovered by xunit2, serialise them to the external test runner, so we don't have to discover them again

[citizenmatt]

Probably worth resurrecting the xunit2-discovery branch (time for another PR?)

[Mpdreamz]

Looking forward to this! If you need someone to test this out let me know.

[BrunoJuchli]

Hi Matt. For a long time i've planned on doing some work on a spec / bdd kind of test tool. I'd like to take advantage of xunit2-discovery mechanisms so that i don't have to implement my own runner, especially r# support, since i guess this must be quite a lot of (recurring) work. I've quite my current job and starting June i hope to have some free time on my hands for open source development. Would you be willing to provide some guidance on implementing xunit2-discovery in resharper-xunit? Maybe we could have a skype chat sometime in June? Thanks for your consideration.

[Red-Folder]

Hi Matt, I currently have a custom Integration Fact (inherits from Fact) with its own Test Case Discoverer. I use an environment variable in the Discoverer to decide if the tests should be included (generally disabled to keep developers using the nimble unit tests locally).

This approach works fine with VS - and the tests appear/ disappear based on the environment variable. However with Resharper they show regardless.

Is that because of this issue? The tests are being discovered due to inheriting from FactAttribute?

Code for that can be found here -> https://github.com/Red-Folder/RedFolder.xUnit.IntegrationFact Thanks

[BrunoJuchli]

@Red-Folder I believe it seems this is not longer being supported anymore, instead, R# supports xunit out of the box, and there's a corresponding bug-entry for this: https://youtrack.jetbrains.com/issue/RSRP-458779#tab=Comments

Sadly enough nothing has changed. On the right hand side you can vote for this issue.

[tullo-x86]

@BrunoJuchli Does this plugin load in R# 2017.x? Because if JetBrains aren't going to implement custom test discovery, someone should.

[BrunoJuchli]

@tullo-x86 Not sure, but I don't think so. btw. @citizenmatt is the maintainer of this plugin as well as the JetBrains Resharper xunit support. Guess they hired him for that ;-)