The field extraData is 97 bytes, but should be 32

[jensenity]

ERROR - The field extraData is 97 bytes, but should be 32. It is quite likely that you are connected to a POA chain. Refer http://web3py.readthedocs.io/en/stable/middleware.html#geth-style-proof-of-authority for more details. The full extraData is: HexBytes('...')

The error happens when I run this code

ethereumetl get_block_range_for_date --provider-uri=https://rinkeby.infura.io --date=2017-04-10

Do we have someone that is already working on a PR that is adding from web3.middleware import geth_poa_middleware & web3.middleware_stack.inject(geth_poa_middleware, layer=0)

to all export codes in cli?

[medvedev1088]

Thanks for reporting this. No one is working on this task yet as far as I know.

[meowsbits]

Have reproduced, while trying to stream from Goerli testnet:

$ ethereumetl stream --start-block 0 --provider-uri file:///tmp/geth-reupholster.ipc -e transaction | tee transactions.csv
Symbolic Execution not available: No module named 'mythril.ether'
2019-12-26 17:10:05,269 - root [INFO] - Using file:///tmp/geth-reupholster.ipc
2019-12-26 17:10:05,310 - root [ERROR] - An exception occurred while syncing block data.
Traceback (most recent call last):
  File "/home/ia/dev/blockchain-etl/ethereum-etl/blockchainetl/streaming/streamer.py", line 77, in _do_stream
    synced_blocks = self._sync_cycle()
  File "/home/ia/dev/blockchain-etl/ethereum-etl/blockchainetl/streaming/streamer.py", line 89, in _sync_cycle
    current_block = self.blockchain_streamer_adapter.get_current_block_number()
  File "/home/ia/dev/blockchain-etl/ethereum-etl/ethereumetl/streaming/eth_streamer_adapter.py", line 38, in get_current_block_number
    return int(Web3(self.batch_web3_provider).eth.getBlock("latest").number)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/eth.py", line 141, in getBlock
    [block_identifier, full_transactions],
  File "/home/ia/.local/lib/python3.6/site-packages/web3/manager.py", line 109, in request_blocking
    response = self._make_request(method, params)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/manager.py", line 92, in _make_request
    return request_func(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/gas_price_strategy.py", line 18, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/attrdict.py", line 18, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/normalize_errors.py", line 9, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/formatting.py", line 57, in apply_formatters
    formatter(response['result']),
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/eth_utils/applicators.py", line 72, in apply_formatter_if
    return formatter(value)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/eth_utils/functional.py", line 45, in inner
    return callback(fn(*args, **kwargs))
  File "/home/ia/.local/lib/python3.6/site-packages/eth_utils/applicators.py", line 84, in apply_formatters_to_dict
    yield key, formatters[key](item)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/validation.py", line 55, in check_extradata_length
    len(result), MAX_EXTRADATA_LENGTH, result
web3.exceptions.ValidationError: The field extraData is 117 bytes, but should be 32. It is quite likely that you are connected to a POA chain. Refer http://web3py.readthedocs.io/en/stable/middleware.html#geth-style-proof-of-authority for more details. The full extraData is: HexBytes('0x22466c6578692069732061207468696e6722202d204166726900000000000000e0a2bd4258d2768837baa26a28fe71dc079f84c70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000')
2019-12-26 17:10:05,313 - root [INFO] - Nothing to sync. Sleeping for 10 seconds...
2019-12-26 17:10:15,328 - root [ERROR] - An exception occurred while syncing block data.
Traceback (most recent call last):
  File "/home/ia/dev/blockchain-etl/ethereum-etl/blockchainetl/streaming/streamer.py", line 77, in _do_stream
    synced_blocks = self._sync_cycle()
  File "/home/ia/dev/blockchain-etl/ethereum-etl/blockchainetl/streaming/streamer.py", line 89, in _sync_cycle
    current_block = self.blockchain_streamer_adapter.get_current_block_number()
  File "/home/ia/dev/blockchain-etl/ethereum-etl/ethereumetl/streaming/eth_streamer_adapter.py", line 38, in get_current_block_number
    return int(Web3(self.batch_web3_provider).eth.getBlock("latest").number)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/eth.py", line 141, in getBlock
    [block_identifier, full_transactions],
  File "/home/ia/.local/lib/python3.6/site-packages/web3/manager.py", line 109, in request_blocking
    response = self._make_request(method, params)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/manager.py", line 92, in _make_request
    return request_func(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/gas_price_strategy.py", line 18, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/attrdict.py", line 18, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/normalize_errors.py", line 9, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/formatting.py", line 57, in apply_formatters
    formatter(response['result']),
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/eth_utils/applicators.py", line 72, in apply_formatter_if
    return formatter(value)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/home/ia/.local/lib/python3.6/site-packages/eth_utils/functional.py", line 45, in inner
    return callback(fn(*args, **kwargs))
  File "/home/ia/.local/lib/python3.6/site-packages/eth_utils/applicators.py", line 84, in apply_formatters_to_dict
    yield key, formatters[key](item)
  File "/home/ia/.local/lib/python3.6/site-packages/web3/middleware/validation.py", line 55, in check_extradata_length
    len(result), MAX_EXTRADATA_LENGTH, result
web3.exceptions.ValidationError: The field extraData is 117 bytes, but should be 32. It is quite likely that you are connected to a POA chain. Refer http://web3py.readthedocs.io/en/stable/middleware.html#geth-style-proof-of-authority for more details. The full extraData is: HexBytes('0x22466c6578692069732061207468696e6722202d204166726900000000000000e0a2bd4258d2768837baa26a28fe71dc079f84c70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000')
2019-12-26 17:10:15,339 - root [INFO] - Nothing to sync. Sleeping for 10 seconds...
^C
Aborted!

[realmeh]

i fix it by: from web3.middleware import geth_poa_middleware self.web3.middleware_onion.inject(geth_poa_middleware, layer=0)

[rphansen91]

@realmeh Can you elaborate on where you made the change?

[xtaci]

has this fixed?

[alexlee85]

tks, @realmeh it works~~~

[rphansen91]

@alexlee85 Where did you make this change?

[rphansen91]

Here is my stack trace after trying to inject the poa middleware

Traceback (most recent call last): File "/ethereum-etl/blockchainetl/streaming/streamer.py", line 77, in _do_stream synced_blocks = self._sync_cycle() File "/ethereum-etl/blockchainetl/streaming/streamer.py", line 89, in _sync_cycle current_block = self.blockchain_streamer_adapter.get_current_block_number() File "/ethereum-etl/ethereumetl/streaming/eth_streamer_adapter.py", line 42, in get_current_block_number Web3.middleware_onion.inject(geth_poa_middleware, layer=0) AttributeError: type object 'Web3' has no attribute 'middleware_onion

I had to use

web3.middleware_stack.inject(geth_poa_middleware, layer=0)

[medvedev1088]

Try web3.middleware_stack.inject(geth_poa_middleware, layer=0)

[jjhesk]

it works! @medvedev1088 @realmeh

[numonedad]

I still see this on the stream subcommand.

[Abhishekkochar]

FYI, this is also happening on the BSC public RPC. Error: Error timestamp: The field extraData is 280 bytes, but should be 32. It is quite likely that you are connected to a POA chain