kazoo
kazoo copied to clipboard
Kazoo hangs in multiprocessing
The following contrived multiprocessing code hangs when run. Kazoo seems to be waiting for its async_set to complete.
#!/usr/bin/python3 from kazoo.client import KazooClient from multiprocessing import Pool
def init_zk_conn(zk_server): kz = KazooClient(zk_server, timeout=10) kz.start() return kz
def update_znode(kz, znode, msg=''): if not kz.exists(znode): kz.create(znode, value=msg.encode('utf-8')) else: kz.set(znode, msg.encode('utf-8'))
KZ = init_zk_conn('localhost:2181')
def update_node(node): znode = '/testkz/' + node update_znode(KZ, znode, 'updating')
if name == 'main': pool = Pool(2) node_list = ['one', 'two'] pool.map(update_node, node_list) print('Done')
I found init Kazoo before fork, and reuse it after fork it will hang. It's looks similar. You may can try (do not init any kazoo client before using and )init their own kazoo in their own process
Any progress on this issue?