python-bitcoinlib icon indicating copy to clipboard operation
python-bitcoinlib copied to clipboard

Multiprocess not working

Open FrancescoSaverioZuppichini opened this issue 4 years ago • 0 comments
trafficstars

Dear all,

I have the following snippet where I just get a block and its transactions using different processes:

from neo4j import GraphDatabase
from decouple import config

import bitcoin
from bitcoin.rpc import RawProxy
import multiprocessing as mp


btc_conf_file = "./bitcoin.conf"
bitcoin.SelectParams(config("CHAIN"))

proxy = RawProxy(btc_conf_file=btc_conf_file)


def pipe(i):
    h = proxy.getblockhash(i)
    b = proxy.getblock(h)
    trans = []
    for tx in b.get("tx", list()):
        t = proxy.getrawtransaction(tx, True, h)
        trans.append(t)
    return h, b, trans


with mp.Pool(8) as pool:

    data = pool.map(pipe, range(1, 125))
    print(data)

And this works great! But if I add proxy.getblockchaininfo()


proxy = RawProxy(btc_conf_file=btc_conf_file)

proxy.getblockchaininfo() # <---- HERE!!

def pipe(i):
   ...
    return h, b, trans


with mp.Pool(8) as pool:

    data = pool.map(pipe, range(1, 125))
    print(data)

I got the following error:

multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/zuppif/anaconda3/envs/bc/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/zuppif/anaconda3/envs/bc/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/zuppif/Documents/Hephaistos/playground.py", line 24, in pipe
    h = proxy.getblockhash(i)
  File "/home/zuppif/anaconda3/envs/bc/lib/python3.8/site-packages/bitcoin/rpc.py", line 361, in <lambda>
    f = lambda *args: self._call(name, *args)
  File "/home/zuppif/anaconda3/envs/bc/lib/python3.8/site-packages/bitcoin/rpc.py", line 266, in _call
    raise JSONRPCError(
bitcoin.rpc.JSONRPCError: {'code': -32700, 'message': 'Parse error'}
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/zuppif/Documents/Hephaistos/playground.py", line 35, in <module>
    data = pool.map(pipe, range(1, 125))
  File "/home/zuppif/anaconda3/envs/bc/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/zuppif/anaconda3/envs/bc/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
bitcoin.rpc.JSONRPCError: {'code': -32700, 'message': 'Parse error'}

Any idea?

Best regards,

Francesco