polkascan-pre-harvester
polkascan-pre-harvester copied to clipboard
Block sync failure on a custom blockchain
We are trying to deploy a polkascan open source version. However we encountered some problems after connected to our node.
Traceback (most recent call last):
File "/usr/src/app/app/tasks.py", line 115, in accumulate_block_recursive
block = harvester.add_block(block_hash)
File "/usr/src/app/app/processors/converters.py", line 469, in add_block
self.process_metadata(parent_spec_version, parent_hash)
File "/usr/src/app/app/processors/converters.py", line 210, in process_metadata
runtime = Runtime.query(self.db_session).get(spec_version)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 959, in get
return self._get_impl(ident, loading.load_on_pk_identity)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 1069, in _get_impl
return db_load_fn(self, primary_key_identity)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 282, in load_on_pk_identity
return q.one()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3292, in one
ret = self.one_or_none()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3261, in one_or_none
ret = list(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3334, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3356, in _execute_and_instances
querycontext, self._connection_from_session, close_with_result=True
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3371, in _get_bind_args
mapper=self._bind_mapper(), clause=querycontext.statement, **kw
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3349, in _connection_from_session
conn = self.session.connection(**kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1124, in connection
execution_options=execution_options,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1130, in _connection_for_bind
engine, execution_options
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 408, in _connection_for_bind
self._assert_active()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 299, in _assert_active
"This session is in 'inactive' state, due to the "
sqlalchemy.exc.InvalidRequestError: This session is in 'inactive' state, due to the SQL transaction being rolled back; no further SQL can be emitted within this transaction.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 375, in trace_task
R = retval = fun(*args, **kwargs)
File "/usr/src/app/app/tasks.py", line 70, in __call__
return super().__call__(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/celery/app/trace.py", line 632, in __protected_call__
return self.run(*args, **kwargs)
File "/usr/src/app/app/tasks.py", line 142, in accumulate_block_recursive
raise HarvesterCouldNotAddBlock(block_hash) from exc
app.processors.converters.HarvesterCouldNotAddBlock: 0xbf62f20192a6d35c59a846d10831ee06275fdcccb179b279832405a804fa5500
If we check the flower console, not every app.tasks.accumulate_block_recursive
task fail.
It looks like related to type_registry. We found types for Kusama, Polkadot, and a substrate-node-template. However we are based on substrate/bin/node
, not that similar to any of the above types. We didn't find a docs to generate the type definition.
So far we have tried:
- Sync to kusama: works
- Sync to our blockchain with kusama config: shows above error
- Sync to our blockchain with node-template config: shows above error
Turned out it's due to a field too short. A change to the table schema is needed. After a quick change, the job is still broken though.
We just pushed on update and probably fix the issues your experiencing, after an update described in https://github.com/polkascan/polkascan-os/blob/master/README.md you can also try the following:
-
Check status of processed blocks at http://127.0.0.1:8080/node-template/harvester/admin If for some reason the harvester gets stuck you can try to give it a 'nudge' by pressing the 'Process' button.
-
Check http://127.0.0.1:8080/node-template/runtime-type if all types are supported. If not, add the custom types to:
harvester/app/type_registry/substrate-node-template.json
I updated the README with some basic troubleshooting: https://github.com/polkascan/polkascan-os#add-custom-types-for-substrate-node-template
Hi I've figured out the root cause.
the root casue is default = sa.Column(sa.String(255)) overflow in our chain (276 length, but varchar max is 255).
Will you accept PR to change this field to TEXT
? our chain base on Flaming fir
which is an another reference implementation in substrate repo