debugger icon indicating copy to clipboard operation
debugger copied to clipboard

Rebasing severly limits the effectiveness of analysis hold during launch

Open xusheng6 opened this issue 5 months ago • 1 comments

A useful feature of the debugger is to "analysis hold" on during debugging. This way, the analysis is limited to the currently visible functions which significantly reduce the likely unnecessary analysis updates. However, when we need to rebase the input binary, the effect is severely limited during launch

To start with, during launch, if we have previously enabled analysis hold, we will have to disable it temporarily (https://github.com/Vector35/debugger/blob/a046bcd862b46d0db280ef6bf292c9443b581bcc/core/debuggercontroller.cpp#L1156). This is because if we do not do so, all of the basic blocks of the function would not be created, and when we pause inside of a function, the debugger is unaware of it (because basic blocks are gone), and it will create a new function at the location.

As a result, we can only turn analysis hold back on after the analysis completes (https://github.com/Vector35/debugger/blob/a046bcd862b46d0db280ef6bf292c9443b581bcc/core/debuggercontroller.cpp#L1171). This works fine, but it kinda goes against the original motivation of introducing the analysis hold -- to suppress as much as re-analysis as possible

I know this probably would not be easy not fix -- a potential improvement is to ensure that during rebasing, "basic" analysis info like basic blocks are not discarded. But that means they need to be updated in place (to account for the new base), which may or may not be easy to do. Also, while having the basic blocks info definitely helps the debugger, the lack of complete analysis info may lead to subtle debugger behavioral differences, e.g., the missing of variables could affect how the debugger annotate them. So it sounds like this couldn't really be resolved until we have a complete overhaul of the rebase? Well I dunno, just dumping my observations and thoughts here for later reference

xusheng6 avatar Jul 08 '25 04:07 xusheng6

When encountering an address with no code created yet, the debugger could probably use the GetPreviousFunctionStartBeforeAddress and GetNextFunctionStartAfterAddress APIs, and then request on demand analysis for those specific functions.

So instead of the debugger creating a function as the first step. Perform the above logic first.

bpotchik avatar Jul 08 '25 11:07 bpotchik