roslyn icon indicating copy to clipboard operation
roslyn copied to clipboard

log data for analyzers/codefixes/refactorings

Open jmarolf opened this issue 2 years ago • 10 comments

draft PR to discuss all the different locations we could/should be logging

jmarolf avatar Nov 02 '22 00:11 jmarolf

So my general feedback is the same as in hte meeting on the topic. Basically, that we don't have confidence that we're recording teh right thing.

CyrusNajmabadi avatar Nov 02 '22 00:11 CyrusNajmabadi

Bring this back to our sight. @CyrusNajmabadi @jmarolf and maybe @ryzngard to get review : ).

Cosifne avatar Nov 29 '22 04:11 Cosifne

Bring this back to our sight. @CyrusNajmabadi @jmarolf and maybe @ryzngard to get review : ).

I can look tomorrow if this is ready. How does this pair with the work that @mavasani did?

ryzngard avatar Nov 29 '22 04:11 ryzngard

Bring this back to our sight. @CyrusNajmabadi @jmarolf and maybe @ryzngard to get review : ).

I can look tomorrow if this is ready. How does this pair with the work that @mavasani did?

I believe Manish's work is measuring the perf of the analyzer (which I think is the time to generate diagnostic data) And here we measure the perf of CodeRefactoring (time to generate the code action) and CodeFix (time to generate code action from diagnostic data) With all these three I think we would have a better granularity of the light bulb showing up time.

Cosifne avatar Nov 29 '22 18:11 Cosifne

i'll look today.

CyrusNajmabadi avatar Nov 29 '22 18:11 CyrusNajmabadi

I think this will work, but I'd love to see some manual testing here to verify what the telemetry looks like compared to user actions.

I will try to see the telemetry locally, and post what I found here today

Cosifne avatar Dec 02 '22 19:12 Cosifne

I think this will work, but I'd love to see some manual testing here to verify what the telemetry looks like compared to user actions.

I will try to see the telemetry locally, and post what I found here today

It seems like this field is different User action: image Trace: image

Cosifne avatar Dec 05 '22 19:12 Cosifne

This appears to be measuring the cost to compute the fix for a particular action. That's certainly useful information. However, my understanding from our discussions on this topic is that we were trying to get information about what teh costs were for showing the lightbulb. Generally speaking, invoking doesn't seem to be an issue. Rather, it's how long it takes for the list to appear. We need telemetry here as if we only track the cost of hte lightbulb itself, that doesn't tell us anything since that's the aggregate cost of all our individual fixers/rfactoring providers.

CyrusNajmabadi avatar Dec 09 '22 01:12 CyrusNajmabadi

This appears to be measuring the cost to compute the fix for a particular action. That's certainly useful information. However, my understanding from our discussions on this topic is that we were trying to get information about what teh costs were for showing the lightbulb. Generally speaking, invoking doesn't seem to be an issue. Rather, it's how long it takes for the list to appear. We need telemetry here as if we only track the cost of hte lightbulb itself, that doesn't tell us anything since that's the aggregate cost of all our individual fixers/rfactoring providers.

(Correct me if I am wrong) My understanding is lightbulb has the ability to show up even when not all the SuggestedActions are ready if we are in async mode. image Are you asking we need to track the time for this list to be poped up? So does it mean to track the cost here?

Cosifne avatar Dec 09 '22 07:12 Cosifne

Are you asking we need to track the time for this list to be poped up? So does it mean to track the cost here?

Yes. But the important part is that we don't care about the total time (that can already be tracked by the editor for example). What we (roslyn) caer about it what is causing hte total time to be that high. For example, a single poorly behaving analyzer or code-fix-provider could make the entire list take 10x as long to show up. We need that information so we can determine which ones are costing so much so we know where to put our efforts into.

CyrusNajmabadi avatar Dec 12 '22 17:12 CyrusNajmabadi