resharper-xunit
resharper-xunit copied to clipboard
xunit
Support asp.net vNext
ReSharper doesn't currently support running tests for asp.net vnext, as it's a new execution environment, and requires a new hosting executable to run the tests via dnx (nee klr), and communicate back to the VS process. This introduces a few moving parts:
- The ReSharper test runner needs to start up the dnx environment, and load a hosting dll
- The hosting dll needs to create a communication channel back to ReSharper, in order to report progress. This is likely to be the same as the communication channel used by ReSharper's current external hosting processes (xml over sockets)
- The hosting dll then needs to load a ReSharper/xunit runner, that can glue xunit and ReSharper together (this is conceptually the same as the current runner dll)
- The xunit runner then has to load the asp.net version of xunit and run the tests, capturing the results, so they can be passed back to ReSharper
I'm in conversation with the ReSharper dev team about getting a dnx host environment launcher working. We obviously want to get something working that will enable debugging, profiling and coverage, too. Once this is in place, I can implement the xunit runner (little known fact, the asp.net environment allows for any test runner, they've only implemented xunit right now).
Does https://youtrack.jetbrains.com/issue/RSRP-435748 mean this work can move forward?
No, that's a separate issue. That issue is about the testing framework for writing a plugin - i.e. making sure your plugin works as it should. To get testing working on dnx (I think that's what we call it now), ReSharper needs to invoke a different executable than it currently does - one that will load CoreCLR and launch an entry point that can communicate back to the main ReSharper process.
My currently half-baked plan (post 9.1 release craziness) is to create a new plugin that can prototype ReSharper/dnx test runner support, and use it in the xunit runner until ReSharper gets native support.
Would this be why the ReSharper extension simply doesn't recognize xUnit tests in a vNext unit test class library? I don't even get as far as trying to run then individually, and invoking the "Run Tests" context menu command simply results in a build.
Yep, this is how it is at the minute - the xunit extension will recognise the tests, and add the icons in the editor gutter, but because there's no support for running a different runtime, the actual running of the tests is broken. It's going to be fixed as part of ReSharper 10.1, due Q1.
That is great news, thanks. I'll be sure to try the plugin as soon as possible after that.
Hi, R# 10.1 EAP is available, but it looks like that it has no dnx runner features. Will this feature moved to the next R# release?
It's only the first EAP. There's no DNX support yet. But then again, there is no DNX any more. Things are still pretty fluid in that world. But it is in progress.
@citizenmatt there's definitely still a DNX, the Dotnet CLI has not been released yet and until that happens (with the rc2-delay and all that) DNX and 1.0.0-rc1-final/update1 bits are still the recommended bit by Microsoft
Are there any rough ETAs on a first stab at running tests in a DNX solution? Currently using the xunit dnx runner integration into the Visual Studio test runner, but it's not R# ;)
Rough ETA? This year :smile:
But seriously folks, I'm working on it right now.
Although I'm uncertain quite what I should target - RC1 or RC2. I'm hoping RC2 comes sooner rather than later, and I can only do the work once. I'll just have to align with where ReSharper's support is, though (I've got some foundational work to do before I can actually look at DNX testing).
Also, is DNX going to remain a thing? I was under the impression that .NET CLI replaced it completely - have I got that wrong?
No dnx will become dotnet in terms of commands at a terminal.
Hopefully, he says, the changes in terms of APIs won't be too large between RC1 & RC2 packages
The terminology is extremely confusing, but technically DNX RC2 was a thing (e.g. 1.0.0-rc2-16357) - it was just short lived because the project ended. But yeah, you can get builds of dotnet/cli from http://dotnet.github.io/ ... dotnet essentially does everything dnx, dnvm, dnu did.
@citizenmatt With the release of RC2 still set to TBD status (and not looking like it's coming anytime soon based upon the last couple of community standups), have you considered targeting RC1 or are you still leaning towards RC2?
I was super excited to read the xUnit support announcement:
http://blog.jetbrains.com/dotnet/2016/03/17/resharper-ultimate-2016-1-formerly-10-1-eap-6/
Until I read the final sentence in the section:
One thing to note is that this build does not yet support running tests for DNX projects, this will hopefully be included in a future build.
Are we still on track to see support for this in 2016.1 or is this feature going to slip?
I'm still on track for 2016.1, although I do need to get a move on. I'm still targeting RC1, as RC2 isn't stable enough (and they've only recently formalised the protocol for running and debugging tests with the new dotnet-test.exe). I'd really like it to be in the next EAP, and we've been updating those weekly.
No. It requires changes to ReSharper itself, and as of the next ReSharper release (2016.1, previously known as 10.1), xunit is now a part of the product, which essentially makes this repo obsolete (I need to update the readme in time for the release!)
I've got ReSharper working with DNX RC1, but for various reasons, we're not shipping it as part of 2016.1. Firstly, there are a few rough edges, as the test support is designed for Visual Studio's runner, which is a different beast to ReSharper. Secondly, and perhaps more importantly, RC1 is already obsolete, and with RC2 being a complete rewrite and just around the corner, we'd end up shipping broken support rather than no support. We'll have an update post RC2 with fixes and the plan is to include the test runner then.
Looks like RC2 is literally right around the corner.
https://blogs.msdn.microsoft.com/dotnet/2016/05/06/net-core-rc2-improvements-schedule-and-roadmap/
Any update on when we're likely to see the tooling support for Resharper?
I'm currently planning on adding it into 2016.2, which we haven't announced a release date for, but should be in the next couple of months - the EAP will open before then, anyway.
We were hoping to be able to update a lot of our .net core/aspnetcore support with RC2, but I'm now worried (from that post) that the tooling isn't going to be final. I'd hate for us to end up in a situation similar to the betas/RC1 where we spent a lot of time chasing breaking changes. We need to know more about the plans for the tooling before we can fully commit. Personally, I'm not rather surprised that we're going to end up with an RTM runtime that doesn't have RTM tooling. There are also some outstanding questions for support that have gone unanswered - dotnet/cli#2501.
So, fingers crossed. I'm still aiming for the 2016.2 release post RC2, but I'm also feeling rather cautious.
Now that .NET Core 1.0 RTM is out, is there any update on the timeline for this? I guess the VS tooling parts aren't RTM yet, but I don't know to what extent that matters for this...
I'm working on it right now. It should be a part of the ReSharper 2016.2 release, which is currently in EAP.
Hi @citizenmatt, I'm wondering if you have an updated ETA at this point? I noticed you just released EAP 10 but I haven't gotten the chance to look at it yet.
Yeah. Unfortunately, it's not going to be part of 2016.2. It's simply not ready in time - the changes touch too many parts (testing itself, but also has an impact on the existing profiling, coverage, continuous testing, support) and it needs more time to stabilise. It will be part of 2016.3, which will open EAP a couple of weeks after 2016.2 is released. It's disappointing, but it's the right thing to do - it's better to delay and release with good quality, than release something that's buggy.