vscode-cpptools icon indicating copy to clipboard operation
vscode-cpptools copied to clipboard

Very long delay when a breakpoint is hit and many (150+) threads are created by the program

Open jusid opened this issue 2 years ago • 71 comments

Environment

  • OS and version: Ubuntu 20.04.5 LTS
  • VS Code: 1.72.1
  • C/C++ extension: v1.12.4
  • OS and version of remote machine (if applicable): gdbserver is running on an embedded ARMv7 Linux
  • GDB version: 7.4

Bug Summary and Steps to Reproduce

Bug Summary: I am developing a C application with 150+ threads and debug it remotely via gdbserver. When the first breakpoint is hit, vscode requests detailed information about every thread in the application and it takes about 3 minutes to complete. The debugging is very painful due to this. The communication between gdb and gdbserver is very sensitive to the network latency. It is important to make as few requests to gdb/gdbserver as possible.

Steps to reproduce: Create an app which starts 150+ threads. Debug it remotely using gdbserver. Set a breakpoint to be hit after all threads are created already. Debug the program. When the breakpoint is hit you will get a significant delay before vscode UI reports the breakpoint hit.

Debugger Configurations

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "(gdb) Debug app",
            "type": "cppdbg",
            "request": "launch",
            "program": "link.out",
            "stopAtEntry": false,
            "cwd": "${workspaceRoot}",
            "environment": [],
            "externalConsole": true,
            "MIMode": "gdb",
            "miDebuggerPath": "arm-linux-gnueabi-gdb",
            "miDebuggerServerAddress": "10.0.1.10:9099",
            "targetArchitecture": "arm",
            "symbolLoadInfo": {
                "loadAll": false
            },
            "logging": {
                "engineLogging": false
            },
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                },
                {
                    "text": "-interpreter-exec console \"set sysroot build_dir/sysroot\"",
                    "ignoreFailures": true
                }
            ]
        }
    ]
}

Debugger Logs

Here is what happens when a break point is hit and why it takes so long. The break point is hit:

1: (20384) ->=thread-created,id="2",group-id="i1"
1: (20384) ->~"[New Thread 1800]\n"
1: (20385) <-1023-thread-info 2
[New Thread 1800]
1: (20389) ->=breakpoint-modified,bkpt={number="3",type="breakpoint",disp="keep",enabled="y",addr="0x00b5efb4",func="x",file="x.c",fullname="x.c",line="568",times="1",original-location="x.c:568"}
1: (20390) ->~"[Switching to Thread 1800]\n"
[Switching to Thread 1800]
1: (20466) ->*stopped,reason="breakpoint-hit",disp="keep",bkptno="3",frame={addr="0x00b5efb4",func="x",args=[],file="x.c",fullname="x.c",line="568"},thread-id="2",stopped-threads="all",core="0"

Information about all(!) threads is requested:

1: (20467) <-1024-thread-info

While GDB executes the request above, it reports availability of new threads (158 in my case):

1: (20509) ->=thread-created,id="3",group-id="i1"
1: (20509) ->~"[New Thread 1639]\n"
1: (20509) ->=thread-created,id="4",group-id="i1"
1: (20509) ->~"[New Thread 1640]\n"
1: (20509) ->=thread-created,id="5",group-id="i1"
1: (20510) ->~"[New Thread 1641]\n"

vscode starts to request info about each reported thread. Even though information about all threads has been requested earlier:

1: (20510) <-1025-thread-info 3
1: (20510) ->=thread-created,id="6",group-id="i1"
1: (20510) ->~"[New Thread 1642]\n"
[New Thread 1639]
1: (20510) ->=thread-created,id="7",group-id="i1"
1: (20510) ->~"[New Thread 1649]\n"
1: (20510) ->=thread-created,id="8",group-id="i1"
...
...
1: (20528) <-1178-thread-info 156
[New Thread 1798]
1: (20528) <-1179-thread-info 157
[New Thread 1799]
1: (20528) <-1180-thread-info 158
[New Thread 1801]
1: (20528) 1023: elapsed time 143
1: (20528) Send Event AD7ThreadCreateEvent
...

Information about all 158 threads has been requested. After 11 seconds GDB returns response to request -thread-info``` and starts to slowly return responses for individual thread info requests:

...
1: (31603) ->1024^done,threads=[{id="158",target-id="Thread 1801",frame={level="0",addr="0xb6e97798",func="??",args=[]},state="stopped",core="0"},{id="157",target-id="Thread 1799",frame={level="0",addr="0xb6fa7ce4",func="??",args=[]},state="stopped",core="0"}, ...
1: (31604) ->(gdb)
1: (31606) 1024: elapsed time 11138
1: (32457) ->1025^done,threads=[{id="3",target-id="Thread 1639",frame={level="0",addr="0xb6e91e7c",func="??",args=[]},state="stopped",core="0"}]
1: (32458) ->(gdb)
1: (32458) 1025: elapsed time 11948
1: (33343) ->1026^done,threads=[{id="4",target-id="Thread 1640",frame={level="0",addr="0xb6e97798",func="??",args=[]},state="stopped",core="0"}]
...

After 151 (!) seconds GDB responds to the last thread info request and vscode is almost ready to inform a user about the break point:

...
1: (171843) ->1180^done,threads=[{id="158",target-id="Thread 1801",frame={level="0",addr="0xb6e97798",func="??",args=[]},state="stopped",core="0"}]
1: (171843) ->(gdb)
1: (171844) 1180: elapsed time 151315
1: (171845) <-1181-stack-list-frames 0 1000

Conclusion: it is not needed to request individual thread info since all thread info is requested anyway and such combined request is executed much much faster compared to many individual requests.

jusid avatar Oct 12 '22 15:10 jusid

You got my vote - got similar problems on our side.

reneparis avatar Oct 13 '22 05:10 reneparis

My vote

olgapshen avatar Oct 14 '22 06:10 olgapshen

you got my vote too, this is a horrible issue that makes vscode much less useful than it should be. We should be able to debug in a timely manner. I'm new to this group. Is there some formal mechanism I can use to show my vote?

jaemzfleming avatar Oct 24 '22 18:10 jaemzfleming

+1 the C++ debugging performance is the major workflow pain our team and colleagues experience using VSCode as a C++ IDE. It gets so bad for me that I sometimes switch to command-line GDB or use GDB in Emacs. Everything else for C++ development in VSCode scales much better in my experience.

rlivings39 avatar Oct 27 '22 18:10 rlivings39

+1

rcontinuum avatar Oct 28 '22 09:10 rcontinuum

+1

gewa24 avatar Nov 10 '22 04:11 gewa24

+1

muddle216 avatar Nov 14 '22 12:11 muddle216

+1

martymcbro avatar Dec 02 '22 10:12 martymcbro

+1

krysgor avatar Dec 02 '22 11:12 krysgor

+1

shaharh9110 avatar Jan 17 '23 16:01 shaharh9110

This has nothing to do with remote debugging. Performance when debugging locally is equally awful.

I have to wait minutes to get local variables to appear (during which time I can do nothing). Closing "locals" in the variables window does nothing because it's automatically reopened when execution next stops.

This effectively makes VSCode unfit for purpose and needs to be fixed.

crazydef avatar Jan 18 '23 11:01 crazydef

+1 why should we have to disable everything just to step normally? Not very useful to me with everything hidden

timmeh87 avatar Jan 23 '23 19:01 timmeh87

+1 from me as well. Very important to have fast gdb on VSCode.

er-mo avatar Feb 14 '23 15:02 er-mo

+1 Just recently started debugging a project again after having it sit around for 2 years and this used to work quite decent and snappy. Now it's almost impossible to debug it at all, I wonder what was introduced in the meantime that caused this

marcomeyerVFX avatar Feb 19 '23 15:02 marcomeyerVFX

+1 I'm currently debugging a multi-threaded program (80+ threads) that takes a long time to step through the lines after hitting a breakpoint. It makes debugging with VS code a rather painful experience. I have no such performance issues when debugging with gdb on the command line

Ubachris avatar Feb 21 '23 06:02 Ubachris

+1 I am debugging on a platform with many threads (~250) and VSCode is now unusable to for this purpose.

phillipmself avatar Mar 02 '23 17:03 phillipmself

Hi all, Is this a related issue? I updated to gdb12 a few months ago and I gotta say the performance seems "nearly reasonable" as a opposed to "it actually literally crashes sometimes". it seems like certain versions of gdb, at least some 7 and some 11, have "bugs" when used with vscode. Check this thread and the post by LeLuc4 for instructions to upgrade to gdb12.

https://github.com/microsoft/vscode-cpptools/issues/103

timmeh87 avatar Mar 02 '23 17:03 timmeh87

@timmeh87 I don't think it's related. I'm using Linux Mint at work, and the version of GDB that ships with is totally broken with VSCode. I had to update to version 12 within a day of getting my machine up and running.

crazydef avatar Mar 02 '23 18:03 crazydef

+1

NikBomb avatar May 10 '23 22:05 NikBomb

For reference: debugging ROS 2 code with Ubuntu 22.04 and the default GDB 12.1 was painfully slow, but with GBD 13.1 performance is quite good.

rjoomen avatar May 15 '23 09:05 rjoomen

Just wanted to address that this issue still persists today on version 1.16.3.

I think that the main problem is an aggressive amount of info threads requests for most events. Maybe there could be some caching mechanism or, at least, a toggle to revert back to a """primitive""" debugger?

kotborealis avatar Jul 18 '23 12:07 kotborealis

I have given up using vscode to debug any serious C++ program, but I still use it as my code editor for C++ programs.

A similar issue has previously been reported in https://github.com/microsoft/vscode-cpptools/issues/2901 4+ years ago, and it is passed to https://github.com/microsoft/vscode/issues/132025 and then pass back here https://github.com/microsoft/vscode-cpptools/issues/9988.

I'm curious about the root cause of this issue. I'm also a programmer, I should have dug deep to find the root cause for myself.

Celthi avatar Jul 22 '23 14:07 Celthi

I am cross debugging an application with 350+ threads and VSCode is because of this issue not the first choice doing this for me.

I understand that there are more important things than that, but an issue that has been open for so long needs to be addressed at some point. Especially if there is no workaround other than not using VSCode.

So pls fix it.

krysgor avatar Aug 11 '23 11:08 krysgor

+1

sb362 avatar Sep 07 '23 13:09 sb362

I'm also affected trying to get familiar with new QT based project DEBUG is useless :( about a minute of waiting if using any of breakpoint, step, continue to line

Version 1.81.1 for Ubuntu

solomax avatar Sep 07 '23 15:09 solomax

+1

yi-shui avatar Sep 18 '23 15:09 yi-shui

+1

egrosspietsch-fds avatar Sep 26 '23 17:09 egrosspietsch-fds

+1

orlandini avatar Oct 05 '23 18:10 orlandini

+1

YongBig avatar Oct 10 '23 05:10 YongBig

Vote. Debugging a large fortran project is infeasible since every step takes several minutes to show thousands of variables. There should be an option to toggle the behavior.

grantyuc avatar Oct 27 '23 04:10 grantyuc