raddebugger icon indicating copy to clipboard operation
raddebugger copied to clipboard

Debugger exceptions from dynamically generated code causes the debugger to pseudo hang.

Open mistymntncop opened this issue 1 year ago • 3 comments

If you run this code it pseudo hangs the debugger. Namely the int3 (0xCC) instruction is never caught by the debugger (no call stack, etc) and the disassembly view becomes "hung" in a forever loading loop. This will be a problem for JITted code.

#include <stdio.h>
#include <stdint.h>
#include <windows.h>

int main(void) {
    printf("1\n");
    
    VOID *code = VirtualAlloc(0, 0x1000, MEM_COMMIT|MEM_RESERVE, PAGE_EXECUTE_READWRITE);
    *((uint32_t*)code) = 0xCCCCCCCC;

    ((void (__fastcall *)()) code)();

    printf("2\n");
    return 0;
}

mistymntncop avatar Jan 23 '24 12:01 mistymntncop

I'm investigating this and it looks like the debugger does catch the exception, although I do see that the call stack is not produced, and the disassembly view does not work. The exception info is displayed in the bottom left-hand corner:

image

Can you confirm this is what you see, or do you not even see this exception info at the bottom?

Just in case it matters, I am building from the top of dev.

ryanfleury avatar Jan 23 '24 22:01 ryanfleury

Hey Ryan. Yes I see the same results as you, the exception is caught according to the info at the bottom (0x80000003).

mistymntncop avatar Jan 23 '24 22:01 mistymntncop

Got it. I've fixed a number of bugs or unsupported behavior for JIT'd code in 804a8406b9f65eaaa621f1dcdee8e57959e8e018 and abb2dd713e5f833c5333e64e975f9a04a9b09bd6. There is one major stumbling block for JIT'd code still, though, which is that the disassembly system/UI do not currently assume that the actual code memory will change, and thus hot-reload when the JIT'd code's memory changes, and so on. I'll let you know when that is fixed.

ryanfleury avatar Jan 23 '24 22:01 ryanfleury

As of 0ead1c034c43e87da1ed107b24ac02ced475de63 (currently just on dev, will be soon merged into master, and will be present in the next release), the debugger's disassembly view has been completely ported over to a more general purpose disassembly visualization layer (e.g. that can also be used in the watch window). This gracefully supports hot-reloading of modified machine code, and so this should work much better with JIT'd code now.

ryanfleury avatar Mar 28 '24 22:03 ryanfleury

Hey @ryanfleury, just letting you know JIT debugging is broken on the latest version (8901dd3). The call stack is missing. If the disassembly view has not loaded anything it will correctly disassemble the JITTed code. However if the dissasembly view already has some other code loaded it will not show the JITTed code.

Same test as before:

#include <stdio.h>
#include <stdint.h>
#include <windows.h>

int main(void) {
    printf("1\n");
    
    VOID *code = VirtualAlloc(0, 0x1000, MEM_COMMIT|MEM_RESERVE, PAGE_EXECUTE_READWRITE);
    *((uint32_t*)code) = 0xCCCCCCCC;

    ((void (__fastcall *)()) code)();

    printf("2\n");
    return 0;
}

mistymntncop avatar May 16 '24 23:05 mistymntncop

This should be re-fixed as of 489ae56223210c3ceefe39a899c34f8b4c9ffb54.

ryanfleury avatar May 23 '24 23:05 ryanfleury