Vector35
CorePluginInit() called twice when the debugger is used as a user plugin
I get the following warning message when I try to use the debugger plugin as a user plugin. I have already turned off corePlugins.debugger so the debugger that comes with Binary Ninja is not loaded.
Setting: debugger.stopAtSystemEntryPoint already registered!
Setting: debugger.stopAtEntryPoint already registered!
Setting: debugger.stackVariableAnnotations already registered!
I added a LogWarn in CorePluginInit(), and I get this:
Native debugger loaded!
Loaded native plugin debuggercore
Native debugger loaded!
Setting: debugger.stopAtSystemEntryPoint already registered!
Setting: debugger.stopAtEntryPoint already registered!
Setting: debugger.stackVariableAnnotations already registered!
Clearly, the function is called twice
The reason that this issue is happening is quite obscure, that when Binary Ninja tries to load the debuggerui as a core plugin, it should fail since the debuggerui does not export a CorePluginInit. However, since the debuggerui is linked with the debuggercore, which happens to have a CorePluginInit, so dlsym will find the init function of the debuggercore, and treat it as the init routine of the debuggerui. This not only makes the debuggerui wrongly treated as a core plugin, it also makes the init routine of the debugger core called twice.
This issue is hard to fix from binary ninja, since there is no way to reliably only looks for symbols in the given module. There is a RTLD_FIRST on macOS that does exactly this, but it is not available on Linux.
So, https://github.com/Vector35/debugger/commit/e2d6489c9b6ebf8a67cc68683afb9b9a340c48bc fixed the issue by changing the debugger build system. After the change, it generates a libdebugger.dylib, which exports the `CorePluginInit. The debugger dylib is just a wrapper and it links against the debugger core. The libdebuggercore.dylib no longer has "CorePluginInit". The debuggerui dylib is unchanged, and it still links aginst the core, which, now does not have the core plugin init function. This way, the libdebugger.dylib will be treated as a core plugin, and the libdebuggerui.dylib is treated as a UI plugin. The libdebuggercore.dylib is NOT treated as a valid plugin.
There is no error message when I load the debugger as a user plugin:
Loaded native plugin debugger
Loaded python3 plugin 'debugger'
Loaded UI plugin debuggerui