PrestoCoverage

PrestoCoverage copied to clipboard

Please add support for Visual Studio 2019.

It should work to be honest. Did you try?

-EDIT You can install it but it wont work. Sorry. when I get some free time I will try and get to it.

unless the installer is locked to vs2017.. otherwise its just a config thing in the installer. ill check

That's the problem, the installer is locked to VS2017 and the Extension gallery within VS2019 does not list your extension.

I installed VS2019 and tested it. There was an issue on installer with dependancy Please use 1.16.4 as there was some dependency that I did not need and is not installed in VS2019 anyway.

I have 1.16.4 installed in 2019 Pro, but get the below error when running MSTest unit tests.

System.TypeAccessException: Attempt by method 'PrestoCoverage.PrestoCoverageCore.OnTestExecutionStarting(System.Object, Microsoft.VisualStudio.TestWindow.Extensibility.OperationStateChangedEventArgs)' to access type 'Microsoft.VisualStudio.TestWindow.Controller.TestRunConfiguration' failed.
   at PrestoCoverage.PrestoCoverageCore.OnTestExecutionStarting(Object sender, OperationStateChangedEventArgs stateArgs) in C:\Projects\PrestoCoverage\PrestoCoverage\PrestoCoverage\PrestoCoverageCore.cs:line 66
   at PrestoCoverage.PrestoCoverageContainerDiscoverer.OperationState_StateChanged(Object sender, OperationStateChangedEventArgs e) in C:\Projects\PrestoCoverage\PrestoCoverage\PrestoCoverage\PrestoCoverageContainerDiscoverer.cs:line 73
   at System.EventHandler`1.Invoke(Object sender, TEventArgs e)
   at Microsoft.VisualStudio.TestWindow.Controller.OperationData.OnStateChanged(OperationStateChangedEventArgs e)
   at Microsoft.VisualStudio.TestWindow.Controller.OperationData.Microsoft.VisualStudio.TestWindow.Controller.IOperationData.OnRequestStateChanged(Object sender, RequestStateChangedEventArgs e)
   at Microsoft.VisualStudio.TestWindow.Controller.Request.SetState(RequestStates newState

Hopefully this helps.

I am looking into this. I noticed this after installing but didnt get round to it yet.

I too get this exact issue. Using 1.16.4 on VS 2019 16.0.3. Unfortunate as I was hoping to use this.

I should note for others as well, you actually need to uninstall the extension to be able to run tests again as disabling it didn't affect the error.

Sorry guys... Something has changed in VS2019 and I am not sure when I will have time to sort it out.

Well I found the problem.. They changed the TestWindows dll in VS2019 so when you try and run a test I try to use reflection to get some information about what is going on with the tests.

Why they did is beyond me but I am too stupid to work out how to get this information using the "Extensions way" because there is no docs anywhere about this. I got this working in 2017 thanks to hours of debugging Visual Studio in Visual Studio and trying all sorts of magic out.

The thing I was using in VS2017, the DLL I had to manually reference had a public sealed class and the 2019 version is internal sealed which prevents me from casting to this type. I can probably go deeper into reflection and get this... but for the love of god.. why?!

#region Assembly Microsoft.VisualStudio.TestWindow.Core, Version=15.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
// C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Microsoft.VisualStudio.TestWindow.Core.dll
#endregion

namespace Microsoft.VisualStudio.TestWindow.Controller
{
    **public** sealed class TestRunConfiguration : RequestConfiguration, ITestRunRequestConfiguration, TestPlatform.ObjectModel.Client.ITestRunConfiguration, Controller.ITestRunConfigurationProvider
    {

#region Assembly Microsoft.VisualStudio.TestWindow.Core, Version=16.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a
// C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Microsoft.VisualStudio.TestWindow.Core.dll
#endregion

namespace Microsoft.VisualStudio.TestWindow.Controller
{
    **internal** sealed class TestRunConfiguration :

So this is why we are getting the error message ... access type 'Microsoft.VisualStudio.TestWindow.Controller.TestRunConfiguration' failed.

Well digging in a bit deeper everything got changed to internal.

If I was conspiracy theorists I would say that one of the Microsoft guys from the Coverlet git saw this and the only reason to close this up to keep this kind of feature for Enterprise users only. But I have no proof of that, I am tired and pissed about this and just speculating. So sorry guys....

No VS2019 support!

Ah man don’t you hate when they do stuff like this? I wonder if it was to prevent not having Enterprise to get these stats. I wonder how they do it in DotCover?

Well if I play the conspiracy card I have to be fair and play the good guy theory card too.

I believe VS2019 is going to force all extensions to run async and they are probably locking down sync access like this down. The thing is the plugin and coverlet run under the VS Thread, luckily its not a massive issue because it doesn't slow down usage while writing code.. and when we run tests we usually go make a brew so nobody notices a minor speed slowdown.

So really.. I have to redo everything to be async and it would be nice if dependency injection worked from VS into extensions (which it doesnt) And all the shit I have seen is still COM hooks, weird export attributes and then my attempt with meta programming.

I have actually stared at the DotCover (and others) source code for hours. I can hardly make head and tails of it but I have literally nicked their VisualStudioExtensions file to help with a few things.

But I just cannot work out how they hook into the Test Window. Usually when I struggle with these things its because I am doing it wrong. But I cannot find any good useful information about Extension developing (like not the shitty examples that avoid using anything useful from VS) 😢

So why do I need the test window? A private object "Configuration" contains the following information about the test runner.

1. Is the debugger attached?
   1. If yes then do not attach Coverlet because coverlet modifes part of the binary and causes debugging to fail
2. Which Test projects are being run?
   1. and iterate over them to give me the exact path of the DLL so I can tell Coverlet what to use

I have spent lots of time trying to get that information from the IDE Interfaces but its just not available. The only reason I found the Configuration object was because I installed Rosyln Analysers that expose private and polymorphic things while debugging... and I spent a lot of time in that shitty little inspector window clicking into layers of properties trying to find something useful. And now they took it away.

I wonder how they do it in DotCover?

I replied before I was staring at the DotCover source code... there is no source code.. I got confused with another open source project AxoCover I think.

The way DotCover does it is by running its own TestRunner out of process. They gather the data then from their own internal gubbins.

Sad thing it didn't work in 2019 since it looks to be a simple and handy extension, but could you remove 2019 from supported versions on the marketplace until this is solved? So people doesn't get confused?

Do you need TestWindow.dll for the basic watch coverage.json files and color gutters functionality? I don't mind running dotnet watch with the appropriate settings myself.

Ahh yes. I completely forgot about that aspect I am so frustrated with this automatic thing. Will try and sort that asap to get it working with 2019 in a limited state at least, until I find some other way to fix the testwindow thing

@ppumkin, you say

So why do I need the test window? A private object "Configuration" contains the following information about the test runner.

Given the trouble we've had with reflection, I have been thinking a bit about possible alternative approaches.

1. Is the debugger attached? i. If yes then do not attach Coverlet because coverlet modifes part of the binary and causes debugging to fail

I eventually realised the question to ask is: How does coverlet itself handle this? Coverlet has a package for direct VSTest DLL integration. Another package is offered for MSBuild integration, but that doesn't help us because the VS test runner uses... VSTest directly.

Now, the coverlet VSTest integration is essentially a DataCollector. The coverlet docs suggest running this collector from the dotnet test command line. But collectors can also be used from .runsettings files, including from Visual Studio.

So IMHO we don't really need any code here. Instead we just need documentation, mainly an example .runsettings file.

2. Which Test projects are being run?

And hopefully the answer here is similar: we need a DataCollector, though this time we write our own, to simply add to our list of recently run test projects. This might require some IPC between the data collector and the main PrestoCoverage instance.

What do you think? Is this worth me or you prototyping?

I know what I am trying to do is probably incorrect. I have learnt about this years ago in development. If it seems too difficult it’s probably because it’s the wrong way 😂

I am a newb to extensions and what you have proposed is a different way. Probably the correct way. So yes please try and make a branch if you will.

All we need to know is to attach coverlet before the test runs... ? And when they finish to aggregate the results.

https://github.com/Microsoft/vstest/tree/master/test/TestAssets/OutOfProcDataCollector

This looks promising. I am sure I have come across before... not sure why I decided to go down reflection route. Probably couldn’t get enough information out of that data collector.

Ah. Well hopefully I have better luck then!

I am amused by your comments about difficult things being the wrong way. Yes. But sometimes wrong code is the only way!

Please try it. I hope you can find a solution.

Good news. This approach works. The code needs tidied up, but I hope to contribute it next week.

Turns out this is bleeding edge stuff. The coverlet DataCollector was only released a week ago, the day I started looking at this! It just wasn't available when you took a look at data collectors.

Ahhh right. Very interesting. I have a feeling your findings will make it a lot easier understand now. But is this stuff is compatible with VS2017 ? Or that can still stay with the fugly reflection garbage?

Cannot wait to see how you fixed it 👍

Ehi guys! Any update on the VS2019 compatibility?

@iainnicol Any news on this issue?

Ah, sorry, mainly I got distracted by other things. I will work on this this weekend.

I had a minor but required pull request rejected for ~~Presto~~ [edit: coverlet] itself, which is unfortunate. I may need a hack. Worst case, I suppose, isn't that bad because ~~Presto~~ [edit: coverlet] has already been forked.

You mean a PR for coverlet right ? Did they for coverlet too?

---

From: Iain Nicol [email protected] Sent: Wednesday, August 14, 2019 8:35:45 PM To: ppumkin/PrestoCoverage [email protected] Cc: Piotr Kula [email protected]; Mention [email protected] Subject: Re: [ppumkin/PrestoCoverage] Support VS 2019 (#17)

Ah, sorry, mainly I got distracted by other things. I will work on this this weekend.

I had a minor but required pull request rejected for Presto itself, which is unfortunate. I may need a hack. Worst case, I suppose, isn't that bad because Presto has already been forked.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ppumkin/PrestoCoverage/issues/17?email_source=notifications&email_token=AAJRDE3T2FHQPAP6QZN7SX3QERGADA5CNFSM4G6VA5TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4JWURA#issuecomment-521366084, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAJRDE667A3OFNVH2MWJKJLQERGADANCNFSM4G6VA5TA.

Oops. I indeed meant coverlet.

We need coverlet to target netstandard (tonerdo/coverlet#470) so that the coverlet VS Test collector works within Visual Studio. Otherwise their VS Test collector only works from the console.

So what to to? Hopefully VS has changed and now works with netcoreapp. If not, maybe we can still consume the coverlet nuget package, but hack the metadata to look like netstandard. Else we have to fork. I know you've had to fork coverlet before, but I don't want to add a second reason for the fork!

What if we fix what we need in our fork. There is nothing changed there really so we could upgrade the pull, upgrade it to net standard ? And PR it back maybe ?

---

From: Iain Nicol [email protected] Sent: Saturday, August 17, 2019 2:50:30 PM To: ppumkin/PrestoCoverage [email protected] Cc: Piotr Kula [email protected]; Mention [email protected] Subject: Re: [ppumkin/PrestoCoverage] Support VS 2019 (#17)

Oops. I indeed meant coverlet.

We need coverlet to target netstandard (tonerdo/coverlet#470https://github.com/tonerdo/coverlet/pull/470) so that the coverlet VS Test collector works within Visual Studio. Otherwise their VS Test collector only works from the console.

So what to to? Hopefully VS has changed and now works with netcoreapp. If not, maybe we can still consume the coverlet nuget package, but hack the metadata to look like netstandard. Else we have to fork. I know you've had to fork coverlet before, but I don't want to add a second reason for the fork!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ppumkin/PrestoCoverage/issues/17?email_source=notifications&email_token=AAJRDE4UB7TKIP47OZHTU63QE7XZNA5CNFSM4G6VA5TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4QKQNA#issuecomment-522233908, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAJRDE2VCBZB6A45CW2ONYTQE7XZNANCNFSM4G6VA5TA.