redis-py
redis-py copied to clipboard
Published
20 hours ago •
redis
redis
Forking Redis Object Bug
Version: redis==4.3.3
Platform: ProductName: Mac OS X ProductVersion: 10.15.7 BuildVersion: 19H1922
Description: Based on my interpretation of the documentation of the ProcessPool class, there was intention to support forking. This does not work as expected for me. Python is unable to serialize the "_lock" object in the ProcessPool, which is a thread lock, not a process lock.
import multiprocessing as mp
import redis
def run_method(redis_pool):
print("hello!")
def run() -> str:
"""Get the username"""
# you can use either of these objects. Creates same error.
# redis_obj = redis.Redis(host='0.0.0.0', port=12345)
redis_obj = redis.ConnectionPool(host="0.0.0.0", port=12345, db=0)
p = mp.Process(target=run_method, args=(redis_obj,))
p.start()
return 'OK'
if __name__ == "__main__":
print(run())
Traceback (most recent call last):
File "/Users/jackhart/Documents/pythonProject/main.py", line 33, in <module>
print(run())
File "/Users/jackhart/Documents/pythonProject/main.py", line 28, in run
p.start()
File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object
Based on my reading this appears closed. Mind re-opening if you feel that this is incorrect?
