"TypeError: cannot pickle '_thread.lock' object" when running simple ProcessSubscriptionProducer on Windows

[henriksod]

Implemented the following class:

import time
import logging
import asyncio

from rtcbot.base import ProcessSubscriptionProducer


class TestProcessProducer(ProcessSubscriptionProducer):

    _log = logging.getLogger("bot.TestProcessProducer")

    def __init__(self, loop=None):

        super().__init__(asyncio.Queue, logger=self._log, loop=loop)

    def _producer(self):
        """
        Runs the actual frame capturing code.
        """

        self._log.info("Started TestProcessProducer")

        self._setReady(True)

        while not self._shouldClose:
            print("PROCESS")
            self._put_nowait("PROCESS")
            time.sleep(1)

        self._setReady(False)
        self._log.info("Ended TestProcessProducer")

which, when instantiating (prod = TestProcessProducer()), produces the following error:

Traceback (most recent call last):
  File "main.py", line 111, in main
    prod = TestProcessProducer()
  File "test_process_producer.py", line 17, in __init__
    super().__init__(asyncio.Queue, logger=self._log, loop=loop)
  File "site-packages\rtcbot\base\multiprocess.py", line 51, in __init__
    self._producerProcess.start()
  File "multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "context.py", line 327, in _Popen
    return Popen(process_obj)
  File "popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object

Is it because I am trying to run this on Windows and Python 3.8? Any way to work around it? Found this ticket where it seems like this is not an issue on Linux only: https://github.com/Koed00/django-q/issues/424

[henriksod]

I ran the same code in WSL and it seems to be working.

The issue is that I am trying to run a non-threadsafe GUI process.

[dkumor]

Many UI toolkits require all UI things to happen in a single thread, which can be a challenge in a multithreaded environment like RTCBot - if that is the issue, then I am not sure if anything can be done in RTCBot itself. Is the problem stemming from the UI toolkit or from RTCBot?

[dkumor]

Sorry, just read up on the bug - I will have to look at this in more detail!

[henriksod]

Hi, I managed to solve it in Linux by creating the following class: https://github.com/dkumor/rtcbot/pull/37

I wanted to create matplotlib plots without blocking the event loop. Matplotlib is not thread safe.