orleans icon indicating copy to clipboard operation
orleans copied to clipboard

Orleans extreme slow when debuging

Open montella1507 opened this issue 1 year ago • 18 comments

Hi,

anyone experiencing issues with debugging on MacOS M2 / Rider?

Orleans is like 500x times slower when debugger is attached.

Orleans 7.0, .NET Core 7.0

montella1507 avatar May 09 '23 17:05 montella1507

When i Detach debugger it runs normally afterward...

montella1507 avatar May 09 '23 18:05 montella1507

Is that 500x number measured or an estimate? Are you attaching a debugger to a Debug build of your application, or a Release build? I would expect it to be about 10x slower, but it depends on what you're doing. I'm also not familiar with the debugger on macOS. Maybe it's much slower.

ReubenBond avatar May 09 '23 18:05 ReubenBond

It may be related to https://github.com/dotnet/runtime/issues/38736 - I've seen this problem crop up in highly parallel, multi threading code, which Orleans naturally leans into.

cmeyertons avatar May 16 '23 00:05 cmeyertons

Seems so! Thank you

Unfortunately i can confirm this:

This is extremely frustrating, and makes debugging barely functional. It's quicker to debug using log statements.

So i hope it will get fixed soon

montella1507 avatar May 16 '23 07:05 montella1507

We're also seeing this with .NET 6.0 + Orleans 3.6.2 and believe that https://github.com/dotnet/runtime/issues/38736 is the root cause. And no, 500x is not an exaggeration.

slonopotamus avatar May 16 '23 16:05 slonopotamus

It sounds like this is a Mac-specific issue. Please👍 that issue and/or respond to it to try to get it resolved. It looks like it has quite a few 'votes' already...

ReubenBond avatar May 16 '23 17:05 ReubenBond

I Will try the performance on Windows then.

montella1507 avatar May 17 '23 20:05 montella1507

@ReubenBond

Windows - JetBrains Rider- 10700k, 64GB RAM, 200 grains:

  • no debug 280 - 330ms
  • debug 1300 ms

macOS - JetBrains Rider- M2 Air , 24GB unified, 200 grains:

  • no debug - 300-370ms
  • debug - 22 000 - 37 000 ms

So it may be mac-specific issue

montella1507 avatar May 25 '23 12:05 montella1507

We also encountered this issue while evaluating Orleans, but for both Windows and MacOS. Really rough to use Orleans like this.

mkuennek avatar Dec 20 '23 13:12 mkuennek

@mkuennek is performance under the debugger equally bad on Win & Mac? When you run under the debugger, are you also changing any other settings, such as log levels? Please add feedback (thumbs up, comment with info) to the original .NET runtime issue: https://github.com/dotnet/runtime/issues/38736

EDIT: also, which debugger are you using on Windows, @mkuennek, is it VS or Rider?

ReubenBond avatar Dec 20 '23 14:12 ReubenBond

@ReubenBond Without having any hard number I would say both Mac and Windows experience the same performance degradation. Both platforms start showing same types of warnings and errors(slow storage access, failing to activate grains). We do not change any other settings. Both runs were in Debug configuration with same log levels, etc.

Thank you for the feedback. Already voted in the mentioned runtime issue.

Both platform tests were done using the current version of Rider but we also had another developer using Windows with VS report this problem.

mkuennek avatar Dec 20 '23 18:12 mkuennek

Thanks for the info, @mkuennek. Hopefully we can get this fixed in the .NET Runtime and/or debuggers. I'm not sure why this is more of an issue on mac than Windows. My inkling was that the issue was related to the way that Rider's debugger handles the notification, but if behavior is the same on Windows under VS (which I have not yet experienced), then perhaps that is not the case. I've asked for more information on the dotnet/runtime thread.

ReubenBond avatar Dec 20 '23 19:12 ReubenBond

I just installed VS to check it myself. The behavior is the same as with Rider.

If you think a small project for reproducing this issue could help, I can try providing one.

mkuennek avatar Dec 20 '23 19:12 mkuennek

Thanks. The dotnet/runtime thread has a minimal repro already. The problem seems to be well understood by the runtime engineers and we're just waiting for a fix now.

ReubenBond avatar Dec 20 '23 19:12 ReubenBond

hello, not on the orleans project, but i have a sort a like issue in our repository (private). I reported this today to Jetbrains: https://youtrack.jetbrains.com/issue/RIDER-104266/Blazor-server-app-performs-very-slow-when-run-from-debugger-on-Mac-M2 , add upvotes or something :D

bc3 avatar Jan 10 '24 07:01 bc3

Adding my vote to the "please address this incredibly important issue" counter.

I just lost several days to this issue, trying to understand why my code was suddenly unusably slow. The batch job I was running takes 3x longer than normal when running under the debugger.

I'm running Rider IDE on a Mac M3 Pro.

yoDon avatar Feb 01 '24 01:02 yoDon

The same thing that keeps me from working.

https://youtrack.jetbrains.com/issue/RIDER-105306/Execution-is-too-slow-when-debugging-code

gaoyang avatar Feb 01 '24 06:02 gaoyang

@gaoyang , @yoDon : i've downloaded EAP JetBrains Rider 2024.1 EAP 5 => see my info in the thread on the jetbrains forum https://youtrack.jetbrains.com/issue/RIDER-104266/Blazor-server-app-performs-very-slow-when-run-from-debugger-on-Mac-M2#focus=Comments-27-9227967.0-0

bc3 avatar Feb 22 '24 07:02 bc3

A fix has just been merged into dotnet/runtime!

ReubenBond avatar May 07 '24 03:05 ReubenBond

For those watching or encountering this, note that the merged fix is for .NET 9 not .NET 8

yoDon avatar May 07 '24 11:05 yoDon

For those watching or encountering this, note that the merged fix is for .NET 9 not .NET 8

Correct. Noah made a comment about it here: https://github.com/dotnet/runtime/pull/101864#issuecomment-2097518279

If you'd like to try this out, you can install a .NET 9.0 nightly build from here: https://github.com/dotnet/installer?tab=readme-ov-file#table

I will close this now, since the runtime issue is fixed. Please feel free to open a new issue referencing this if you hit similar symptoms.

ReubenBond avatar May 15 '24 18:05 ReubenBond