dora does not handle concurrent call in python

[haixuanTao]

Discussed in https://github.com/orgs/dora-rs/discussions/746

^{Originally posted by jrabary January 6, 2025} Hi all,

I'm trying to add an interactive and realtime visualisation node in my graph. I end up having the following events handlers in the GUI

    def handle_submit(command):
        if command != "":
            instruction_array = pa.array([command])
            node.send_output("instruction", instruction_array)
        return ""

    def handle_camera_streams():
        current_im = im
        while True:
            evt = node.next(timeout=FPS)
            if evt is not None:
                if evt["type"] == "INPUT":
                    if evt["id"] == "observations":
                        obs = decode_dict(evt["value"])
                        current_im = obs["overhead_cam"]

            yield current_im

The first one is to handle user action on the GUI and the second on to display a video stream on the GUI. The point is, the two handlers can work separatly but once I put them together I got some errors

node.send_output("instruction", instruction_array)
RuntimeError: Already borrowed

I suspect dora doesn't like I use the same node object concurrently.

Any ideas ?

[7SOMAY]

Hi @haixuanTao, I believe this issue can be handled using multi-threading in Python. We can create separate threads for each functionality to process them simultaneously without blocking.

I guess this will work, what do you think?

[haixuanTao]

I think that we currently have an issue if we try to call in parallel dora from different thread, but haven´t really investigated deeply with this.

[7SOMAY]

I think that we currently have an issue if we try to call in parallel dora from different thread, but haven´t really investigated deeply with this.

Oh I see, if you want-I can give it a try. Just let me know the reference and a kick start to grasp the exact situation to work on.

[haixuanTao]

Honestly. it's quite an intricate issue as it is deeply rooted within the mixed rust-python side of dora.

[7SOMAY]

Honestly. it's quite an intricate issue as it is deeply rooted within the mixed rust-python side of dora.

Just let me have a quick overview of that, I’ll try to get my hands on that. May be it will enlighten me a bit more about dora because I’ve worked on concurrency in past.

But only if you say so, o/w I’ll keep working on my existing issues.

[haixuanTao]

Being bluntly honest, I don't even know myself how to fix it! XD

I'm not even sure where does this come from. I genuinely think that it can take about a week to fix

[7SOMAY]

Okay got it, if you find any lead to this. Please raise detailed descriptions, till then I'll work on pending issues and try to get a little bit of understanding of this issue as well :)

[haixuanTao]

So this issue is very clearly linked to Data attached to pyclass instances is protected from concurrent access by a RefCell-like pattern of runtime borrow checking https://pyo3.rs/v0.26.0/free-threading.html#exceptions-and-panics-for-multithreaded-access-of-mutable-pyclass-instances

[haixuanTao]

This PR: https://github.com/oortlieb/dora/pull/1/files thanks to @oortlieb makes it possible to run concurrent call for python dora at the expense of introduce an Arc<Mutex> which could honnestly be not that big of a deal if dora is fast enough.

This could make things like #1238 much easier to do as we could just pass node as an argument and then "not worry" about concurrency too much.

Open for discussion