kazoo icon indicating copy to clipboard operation
kazoo copied to clipboard

Kazoo hangs in multiprocessing

Open ZedMcChen opened this issue 6 years ago • 2 comments

The following contrived multiprocessing code hangs when run. Kazoo seems to be waiting for its async_set to complete.

#!/usr/bin/python3 from kazoo.client import KazooClient from multiprocessing import Pool

def init_zk_conn(zk_server): kz = KazooClient(zk_server, timeout=10) kz.start() return kz

def update_znode(kz, znode, msg=''): if not kz.exists(znode): kz.create(znode, value=msg.encode('utf-8')) else: kz.set(znode, msg.encode('utf-8'))

KZ = init_zk_conn('localhost:2181')

def update_node(node): znode = '/testkz/' + node update_znode(KZ, znode, 'updating')

if name == 'main': pool = Pool(2) node_list = ['one', 'two'] pool.map(update_node, node_list) print('Done')

ZedMcChen avatar Aug 14 '18 12:08 ZedMcChen

I found init Kazoo before fork, and reuse it after fork it will hang. It's looks similar. You may can try (do not init any kazoo client before using and )init their own kazoo in their own process

ShireCiel avatar Feb 21 '19 09:02 ShireCiel

Any progress on this issue?

hungrybirder avatar Dec 31 '19 10:12 hungrybirder