Performance issue with macOS remote-debugging

[xusheng6]

I noticed two issues that cause a significant slow down during launch while during macOS debugging.

1. The file being debugging is always copied to the remote system despite the file with the same MD5 already exists on the remote system.
2. LLDB reads libobjc.A.dylib from memory. This is evident by the warning message during launch:

libobjc.A.dylib is being read from process memory. This indicates that LLDB could not find the on-disk shared cache for this device. This will likely reduce debugging performance.

I suppose this is caused by the shared cache mechanism on macOS, and libobjc.A.dylib does not exist on disk as a single file, so LLDB does not have a way to read it from disk. So it has to read it from memory. This causes a significantly slow down, often times to the scale of 20+ seconds. It appears that reading from the memory is also slower, potentially because once the file gets mapped, it takes more memory bytes (e..g, the .bss section will actually take space now)

[xusheng6]

For 1), this is related to LLDB's "installation" of the main executable, and running settings set target.auto-install-main-executable false can avoid it. But the launch will also not work if the file is not already there

[xusheng6]

For 2), I tried a few ways to avoid it, but so far it does not make a huge difference

A useful way of troubleshooting is to run log enable --file ~/Downloads/lldb-gdbremote.log --timestamp gdb-remote packets, which will dump all of the gdb rsp packets. Among them, packets with vFile are related to file operations, and vFile:open specifically shows all of the file that will be fetched

[longxk]

1. this command settings set target.auto-install-main-executable false can prevent unnecessary file copying, but "Initial LLDB Command" setting field seems accept only one command, how do i run multiples init commands?
2. got a 450M log file by just launching Hex Fiend.app in remote debugging mode.

lldb-gdbremote.log.zip

[xusheng6]

1. this command settings set target.auto-install-main-executable false can prevent unnecessary file copying, but "Initial LLDB Command" setting field seems accept only one command, how do i run multiples init commands?

   2. got a 450M log file by just launching Hex Fiend.app in remote debugging mode.

lldb-gdbremote.log.zip

I will look into running multiple commands

The size of the log file I see is 95M, not 450M, did you redact something? (It is fine, but I am just curious about the difference

Also -- how did you manage to debug Hex Fiend? Did you turn off SIP or resigned it?

[longxk]

1. this command settings set target.auto-install-main-executable false can prevent unnecessary file copying, but "Initial LLDB Command" setting field seems accept only one command, how do i run multiples init commands?

   2. got a 450M log file by just launching Hex Fiend.app in remote debugging mode.

lldb-gdbremote.log.zip

I will look into running multiple commands

The size of the log file I see is 95M, not 450M, did you redact something? (It is fine, but I am just curious about the difference

Also -- how did you manage to debug Hex Fiend? Did you turn off SIP or resigned it?

1. the log file is too large for uploading, so I removed part of it in the middle.
2. SIP is off on the remote machine.

[xusheng6]

I have created https://github.com/Vector35/debugger/issues/823 and https://github.com/Vector35/debugger/issues/824 out of this issue

[xusheng6]

I got a local repro of this, will look into this later!

[xusheng6]

I think the primary reason is that LLDB is reading a ton of memory bytes from the remote target. There are 72MB of data read, but it is reading it in 0x200 bytes chunk, so it is very slow. I am not exactly sure why is that happening, but perhaps it is doing its own analysis (e.g., symbol info)