json.decoder.JSONDecodeError

[mousewu]

I am trying to extract all transactions from bitcoin blockchain using the script below: bitcoinetl export_all --start 505200 --end 593600 --partition-batch-size 100 --provider-uri http://localhost:8332 --chain bitcoin --output-dir /data1/bitcoinetl/ --max-workers 10 --export-batch-size 100 --enrich True

But at the block number range of 505200 - 505299, the following error consistently occurs: File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/click/core.py", line 764, in call return self.main(*args, **kwargs) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/click/core.py", line 717, in main rv = self.invoke(ctx) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/click/core.py", line 1137, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/click/core.py", line 956, in invoke return ctx.invoke(self.callback, **ctx.params) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/click/core.py", line 555, in invoke return callback(*args, **kwargs) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/cli/export_all.py", line 102, in export_all output_dir, provider_uri, max_workers, export_batch_size, enrich) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/jobs/export_all.py", line 117, in export_all job.run() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/blockchainetl/jobs/base_job.py", line 30, in run self._end() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/jobs/enrich_transactions.py", line 103, in _end self.batch_work_executor.shutdown() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/blockchainetl/executors/batch_work_executor.py", line 68, in shutdown self.executor.shutdown() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/blockchainetl/executors/fail_safe_executor.py", line 39, in shutdown self._check_completed_futures() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/blockchainetl/executors/fail_safe_executor.py", line 47, in _check_completed_futures future.result() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/concurrent/futures/_base.py", line 425, in result return self.__get_result() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result raise self._exception File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/blockchainetl/jobs/base_job.py", line 28, in run self._export() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/jobs/enrich_transactions.py", line 54, in _export self.batch_work_executor.execute(self.transactions_iterable, self._enrich_transactions) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/blockchainetl/executors/batch_work_executor.py", line 48, in execute self.executor.submit(self._fail_safe_execute, work_handler, batch) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/blockchainetl/executors/fail_safe_executor.py", line 31, in submit self._check_completed_futures() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/blockchainetl/executors/fail_safe_executor.py", line 47, in _check_completed_futures future.result() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/concurrent/futures/_base.py", line 425, in result return self.__get_result() File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result raise self._exception File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/concurrent/futures/thread.py", line 56, in run result = self.fn(*self.args, **self.kwargs) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/blockchainetl/executors/batch_work_executor.py", line 64, in _fail_safe_execute work_handler([item]) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/jobs/enrich_transactions.py", line 63, in _enrich_transactions input_transactions_map = self._get_input_transactions_as_map(transaction_input_batch) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/jobs/enrich_transactions.py", line 82, in _get_input_transactions_as_map transactions = self.btc_service.get_transactions_by_hashes(transaction_hashes) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/service/btc_service.py", line 96, in get_transactions_by_hashes raw_transactions = self._get_raw_transactions_by_hashes_batched(hashes) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/service/btc_service.py", line 121, in _get_raw_transactions_by_hashes_batched result.extend(self._get_raw_transactions_by_hashes(batch)) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/service/btc_service.py", line 134, in _get_raw_transactions_by_hashes transaction_detail_response = self.bitcoin_rpc.batch(transaction_detail_rpc) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/rpc/bitcoin_rpc.py", line 48, in batch response = self._decode_rpc_response(raw_response) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/site-packages/bitcoinetl/rpc/bitcoin_rpc.py", line 71, in _decode_rpc_response return json.loads(response_text, parse_float=decimal.Decimal) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/json/init.py", line 367, in loads return cls(**kw).decode(s) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/data/python/anaconda3/envs/blockchain-etl/lib/python3.6/json/decoder.py", line 355, in raw_decode obj, end = self.scan_once(s, idx) json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 134037348 (char 134037347)

Does anyone know how to fix it?

[akhorshidi]

I've encountered the same issue:

stream --provider-uri http://user:pass@bitcoind:8332 --chain bitcoin --last-synced-block-file config/last_synced_block.txt --period-seconds 30 --batch-size 40 --block-batch-size 120 --max-workers 20 --output path_to_dir

However using the default values, I see no problem! stream --provider-uri http://user:pass@bitcoind:8332 --chain bitcoin --last-synced-block-file config/last_synced_block.txt --output path_to_dir