Socket not closed, when GhidraBridge Object gets out of scope

[maxeisele]

I have observed, that the socket to ghidra is not closed, when the GhidraBridge Object gets out of scope. It gets closed, when the whole python process terminates. We have a long running python process and do some ghidra operations from time to time via a remotifyied class:

with ghidra_bridge.GhidraBridge(  ) as bridge:
            RemoteGhidraClass = bridge.remoteify(RemoteGhidra)
            remote_ghidra = RemoteGhidraClass().do_something()

In the log from Ghidra, we can see WARNING:jfx_bridge.bridge:Handling connection from ('127.0.0.1', 55500) each time the above code is called (with different port of course), but the connection gets not closed. Also we experience some hung ups now and then. Could that be related? We have also experienced hangs, when keeping the GhidraBridge Object alive across the different calls. What would be the correct was here?

At the same time, we are experiencing following error log:

[ERROR bridge.py:522 run()] 'utf-8' codec can't decode byte 0xaa in position 327679: invalid start byte
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/jfx_bridge/bridge.py", line 504, in run
    msg_dict = json.loads(data.decode("utf-8"))
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaa in position 327679: invalid start byte

[justfoxing]

I'm assuming that decode error is from the same code as the user who opened justfoxing/jfx_bridge#17 , yeah? If not, would love to see your code as well. Let's handle the debugging for that one in that issue.

The exact time when the bridge socket gets closed is a bit messy - it happens when there's no references left to the bridge, and any bridged object will maintain a reference. If you're keeping something like remote_ghidra to use outside the block, the bridge it came over will stay alive as well. (Also, it's entirely possible I left bugs here - there's a strong potential for cyclic dependencies that would result in things living forever).

It sounds like there's a range of issues here, with not enough info for me to see what's going on. My recommendation would be to not create multiple ghidra_bridges - instead, alter your code so that the first time you create a bridge, you keep it around to use for subsequent operations. Then, if you have issues after that, I'd love to be able to see your code as well as the error - that'd help me be able to give better advice.

[maxeisele]

First, thanks for your fast reply

I'm assuming that decode error is from the same code as the user who opened justfoxing/jfx_bridge#17 , yeah? If not, would love to see your code as well. Let's handle the debugging for that one in that issue.

Right, some communication error.

The exact time when the bridge socket gets closed is a bit messy - it happens when there's no references left to the bridge, and any bridged object will maintain a reference. If you're keeping something like remote_ghidra to use outside the block, the bridge it came over will stay alive as well. (Also, it's entirely possible I left bugs here - there's a strong potential for cyclic dependencies that would result in things living forever).

I will refactor the code such, that I will keep one ghidra bridge alive and see if I still experience these hangs. If yes, I will try to provide code to reproduce it.

[maxeisele]

Regarding, the opened sockets. If you execute this code multiple times in a jupyter notebook, the sockets are note closed, and a new one is opened on each execution.

import ghidra_bridge
import logging as log

def long_runing_function():
    ret = []
    for i in range(100):
        ret.append(i)

    return ret



bridge = ghidra_bridge.GhidraBridge(
                response_timeout=3600,
                interactive_mode=False,
                loglevel=log.DEBUG
        )
remote_function = bridge.remoteify(long_runing_function)

ret = remote_function()

Let's discuss the decode error in issue justfoxing/jfx_bridge#17

[justfoxing]

Ah, right, this is an issue with jupyter that's come up before (see https://github.com/justfoxing/ghidra_bridge/issues/50). I think it might be to do with the underlying Ipython kernel retaining output history (the _# output variables recorded everytime a command/cell returns an output). This prevents those objects and anything they reference (including bridged objects) from actually being deleted (and the bridge connection stays open while any bridged objects it created are still around).

Only workaround I've identified was to restart the kernel to close the sockets.

If you place the bridge = ... line in its own cell at the top of the notebook and avoid rerunning that, subsequent cells can be rerun using that one bridge without having to create a new bridge each time.

[justfoxing]

Alternatively, you could try wrapping the bridge creation in something a little like the following, so that it only ever gets run once, regardless of how many times its cell is rerun.

try:
    if bridge is None:
        raise Exception()
except Exception:
    bridge = ghidra_bridge.GhidraBridge(...)