pendulum Parachain stopped finalizing blocks after upgrade

I am running a pendulum rpc node and after upgrading from https://github.com/pendulum-chain/pendulum/releases/tag/1.1.0 to https://github.com/pendulum-chain/pendulum/releases/tag/1.6.0 the node stopped finalizing blocks and produced the errors in the log below

The container in which it runs is on ubuntu 22.04 and run with the following flags

./pendulum-collator \
--name="\U0001F6E1 DWELLIR PENDULUM RPC \U0001F6E1" \
--chain=pendulum \
--pruning=archive \
--rpc-max-connections=1000 \
--rpc-external --rpc-port=9933 \
--rpc-cors=all \
--rpc-methods=unsafe \
--prometheus-external \
--runtime-cache-size=16 \
--max-runtime-instances=32 \
--execution wasm \
-- \
--chain=polkadot

Node logs

Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: 2025-02-04 16:11:15 [Parachain] Evicting failed runtime instance error=Runtime panicked: Could not lookup information required to validate the transaction
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: 2025-02-04 16:11:15 [Parachain] Block prepare storage changes error: Error at calling runtime api: Execution failed: Runtime panicked: Could not lookup information required to validate the transaction
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: 2025-02-04 16:11:15 [Parachain] 💔 Error importing block 0x7f9123dfff8be6108cd59a5b3346dbcbb6e1f0a54f5c0bde7c140db14f9d2466: consensus error: Import failed: Error at calling runtime api: Execution failed: Runtime panicked: Could not lookup information required to validate the transaction
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: ====================
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: Version: 1.6.0-93c9dbe0542
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    0: sp_panic_handler::set::{{closure}}
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    1: std::panicking::rust_panic_with_hook
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    2: std::panicking::begin_panic_handler::{{closure}}
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    3: std::sys_common::backtrace::__rust_end_short_backtrace
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    4: rust_begin_unwind
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    5: core::panicking::panic_fmt
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    6: frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::execute_extrinsics_with_book_keeping::{{closure}}::panic_cold_display
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    7: frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::execute_extrinsics_with_book_keeping
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    8: tracing::span::Span::in_scope
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:    9: frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::execute_block
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   10: amplitude_runtime::api::dispatch
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   11: environmental::using
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   12: sc_executor::executor::WasmExecutor<H>::with_instance::{{closure}}
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   13: sc_executor::wasm_runtime::RuntimeCache::with_instance
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   14: <sc_executor::executor::NativeElseWasmExecutor<D> as sp_core::traits::CodeExecutor>::call
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   15: sp_state_machine::execution::StateMachine<B,H,Exec>::execute
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   16: <sc_service::client::call_executor::LocalCallExecutor<Block,B,E> as sc_client_api::call_executor::CallExecutor<Block>>::contextual_call
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   17: <sc_service::client::client::Client<B,E,Block,RA> as sp_api::CallApiAt<Block>>::call_api_at
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   18: <amplitude_runtime::RuntimeApiImpl<__SrApiBlock__,RuntimeApiImplCall> as sp_transaction_pool::runtime_api::TaggedTransactionQueue<__SrApiBlock__>>::__runtime_api_internal_call_api_at
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   19: sp_api::Core::execute_block
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   20: <&sc_service::client::client::Client<B,E,Block,RA> as sc_consensus::block_import::BlockImport<Block>>::import_block::{{closure}}
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   21: <alloc::sync::Arc<T> as sc_consensus::block_import::BlockImport<B>>::import_block::{{closure}}
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   22: <cumulus_client_consensus_common::ParachainBlockImport<Block,BI,BE> as sc_consensus::block_import::BlockImport<Block>>::import_block::{{closure}}
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   23: <alloc::boxed::Box<dyn sc_consensus::block_import::BlockImport<B>+Error = sp_consensus::error::Error+core::marker::Send+core::marker::Sync> as sc_consensus::block_import::BlockImport<B>>::import_block::{{closure}}
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   24: futures_util::future::future::FutureExt::poll_unpin
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   25: sc_consensus::import_queue::basic_queue::BlockImportWorker<B>::new::{{closure}}
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   26: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   27: <sc_service::task_manager::prometheus_future::PrometheusFuture<T> as core::future::future::Future>::poll
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   28: <futures_util::future::select::Select<A,B> as core::future::future::Future>::poll
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   29: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   30: tokio::runtime::park::CachedParkThread::block_on
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   31: tokio::runtime::handle::Handle::block_on
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   32: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   33: tokio::runtime::task::core::Core<T,S>::poll
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   34: tokio::runtime::task::harness::Harness<T,S>::poll
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   35: tokio::runtime::blocking::pool::Inner::run
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   36: std::sys_common::backtrace::__rust_begin_short_backtrace
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   37: core::ops::function::FnOnce::call_once{{vtable.shim}}
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   38: std::sys::pal::unix::thread::Thread::new::thread_start
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   39: <unknown>
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:   40: <unknown>
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: Thread 'tokio-runtime-worker' panicked at 'Could not lookup information required to validate the transaction', /home/gitlab-runner/.cargo/git/checkouts/polkadot-sdk-2a8e7ffb960df38b/0a31531/substrate/frame/executive/src/lib.rs:578
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: This is a bug. Please report it at:
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]:         https://github.com/pendulum-chain/pendulum/issues/new
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: 2025-02-04 16:11:15 [Parachain] Failed to check if the parachain block expired
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: 2025-02-04 16:11:15 [Parachain] Evicting failed runtime instance error=Runtime panicked: Could not lookup information required to validate the transaction
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: 2025-02-04 16:11:15 [Parachain] Block prepare storage changes error: Error at calling runtime api: Execution failed: Runtime panicked: Could not lookup information required to validate the transaction
Feb 04 16:11:15 juju-ab466c-0 polkadot[2284297]: 2025-02-04 16:11:15 [Parachain] 💔 Error importing block 0x7f9123dfff8be6108cd59a5b3346dbcbb6e1f0a54f5c0bde7c140db14f9d2466: consensus error: Import failed: Error at calling runtime api: Execution failed: Runtime panicked: Could not lookup information required to validate the transaction

Feb 04 '25 16:02 gflow33

Same with upgrade from pendulum-collator 1.1.0 to 1.6.0-a on Ubuntu 22.04 LTS. Used binary from git

Feb 16 '25 19:02 stakeworks

Hey guys @gflow33 and @stakeworks, sorry for the late reply. We are still figuring out some bugs with the new node binary. Release 1.6.0-a was supposed to fix some issues around the warp-sync not working anymore. We are still seeing the node unable to connect to other peers of the relay chain in our environment.

For the time being, please stay on the collator binaries for 1.1.0. We are also still using them for our own collators. I'll reach out here and on the Pendulum Discord server once you can upgrade your node binaries again.

Feb 17 '25 09:02 ebma

Hey @gflow33 and @stakeworks. We have finally resolved the issues in the collator binaries, allowing them to work with the latest runtime. Please consider upgrading your nodes to 1.6.0-d. If you have any other issues running them, please let us know. I'll close this issue already but we can re-open it if there are new issues.

Aug 04 '25 09:08 ebma

Upgrade 1.1.0 -> 1.6.0-d also upgrade from 1.6.0-b -> 1.6.0-d 2025-08-04 09:59:00 💾 Database: RocksDb at /home/pendulum/.pendulum-collator/chains/pendulum/db/full Error: Service(Other(""Invalid JSON blob: unknown variant native, expected one of Native, XCM, Stellar, ZenlinkLPToken, Token at line 1 column 52"")) pendulum-collator.service: Main process exited, code=exited, status=1/FAILURE pendulum-collator.service: Failed with result 'exit-code'.

Downgrade 1.6.0-d -> 1.6.0-a 2025-08-04 10:06:49 [Parachain] panicked at /home/builder/cargo/git/checkouts/polkadot-sdk-2a8e7ffb960df38b/0a31531/cumulus/pallets/parachain-system/src/lib.rs:1339:3: no space left for the block in the unincluded segment 2025-08-04 10:06:49 [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back. 2025-08-04 10:06:49 [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back. 2025-08-04 10:06:49 [Parachain] ❗️ Inherent extrinsic returned unexpected error: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm unreachable instruction executed WASM backtrace: error while executing at wasm backtrace: 0: 0x75b494 - pendulum_runtime.wasm!rust_begin_unwind 1: 0xc93a - pendulum_runtime.wasm!core::panicking::panic_fmt::hac62c46e7ac26bea 2: 0x3d4412 - pendulum_runtime.wasm!cumulus_pallet_parachain_system::<impl cumulus_pallet_parachain_system::pallet::Pallet<T>>::maybe_drop_included_ancestors::h0ca6ac067ab071b1 3: 0x66f91d - pendulum_runtime.wasm!frame_support::storage::transactional::with_transaction::h7056de9b958d3a12 4: 0x2b27b6 - pendulum_runtime.wasm!<cumulus_pallet_parachain_system::pallet::Call<T> as frame_support::traits::dispatch::UnfilteredDispatchable>::dispatch_bypass_filter::{{closure}}::heb5b0ab65d3355fd 5: 0x2b8105 - pendulum_runtime.wasm!environmental::local_key::LocalKey<T>::with::h55450cbda69d64e5 6: 0x9d1db - pendulum_runtime.wasm!<pendulum_runtime::RuntimeCall as frame_support::traits::dispatch::UnfilteredDispatchable>::dispatch_bypass_filter::h2e8ebd3a47e1e452 7: 0x6d3036 - pendulum_runtime.wasm!<sp_runtime::generic::checked_extrinsic::CheckedExtrinsic<AccountId,Call,Extra> as sp_runtime::traits::Applyable>::apply::h8aff7ef5e9718e49 8: 0x2e93a6 - pendulum_runtime.wasm!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::apply_extrinsic::h5a50b7fbfdc60171 9: 0x3e0167 - pendulum_runtime.wasm!BlockBuilder_apply_extrinsic. Dropping. 2025-08-04 10:06:49 [Parachain] panicked at /home/builder/cargo/git/checkouts/polkadot-sdk-2a8e7ffb960df38b/0a31531/cumulus/pallets/parachain-system/src/lib.rs:265:6: set_validation_data inherent needs to be present in every block! 2025-08-04 10:06:49 [Parachain] err=Error { inner: Proposing Caused by: 0: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm unreachable instruction executed WASM backtrace: error while executing at wasm backtrace: 0: 0x75b494 - pendulum_runtime.wasm!rust_begin_unwind 1: 0xc93a - pendulum_runtime.wasm!core::panicking::panic_fmt::hac62c46e7ac26bea 2: 0xfe9c - pendulum_runtime.wasm!core::option::expect_failed::hc18f070396e13242 3: 0x379a19 - pendulum_runtime.wasm!<(TupleElement0,TupleElement1,TupleElement2,TupleElement3,TupleElement4,TupleElement5,TupleElement6,TupleElement7,TupleElement8,TupleElement9,TupleElement10,TupleElement11,TupleElement12,TupleElement13,TupleElement14,TupleElement15,TupleElement16,TupleElement17,TupleElement18,TupleElement19,TupleElement20,TupleElement21,TupleElement22,TupleElement23,TupleElement24,TupleElement25,TupleElement26,TupleElement27,TupleElement28,TupleElement29,TupleElement30,TupleElement31,TupleElement32,TupleElement33,TupleElement34,TupleElement35,TupleElement36,TupleElement37,TupleElement38,TupleElement39,TupleElement40,TupleElement41,TupleElement42,TupleElement43,TupleElement44,TupleElement45,TupleElement46,TupleElement47,TupleElement48,TupleElement49,TupleElement50,TupleElement51,TupleElement52,TupleElement53,TupleElement54) as frame_support::traits::hooks::OnFinalize<BlockNumber>>::on_finalize::h185b0edbdb72401e 4: 0x2e981b - pendulum_runtime.wasm!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::idle_and_finalize_hook::h923d32de48ac8de7 5: 0x3e02c3 - pendulum_runtime.wasm!BlockBuilder_finalize_block 1: Execution failed: Execution aborted due to trap: wasm trap: wasm unreachable instruction executed WASM backtrace: error while executing at wasm backtrace: 0: 0x75b494 - pendulum_runtime.wasm!rust_begin_unwind 1: 0xc93a - pendulum_runtime.wasm!core::panicking::panic_fmt::hac62c46e7ac26bea 2: 0xfe9c - pendulum_runtime.wasm!core::option::expect_failed::hc18f070396e13242 3: 0x379a19 - pendulum_runtime.wasm!<(TupleElement0,TupleElement1,TupleElement2,TupleElement3,TupleElement4,TupleElement5,TupleElement6,TupleElement7,TupleElement8,TupleElement9,TupleElement10,TupleElement11,TupleElement12,TupleElement13,TupleElement14,TupleElement15,TupleElement16,TupleElement17,TupleElement18,TupleElement19,TupleElement20,TupleElement21,TupleElement22,TupleElement23,TupleElement24,TupleElement25,TupleElement26,TupleElement27,TupleElement28,TupleElement29,TupleElement30,TupleElement31,TupleElement32,TupleElement33,TupleElement34,TupleElement35,TupleElement36,TupleElement37,TupleElement38,TupleElement39,TupleElement40,TupleElement41,TupleElement42,TupleElement43,TupleElement44,TupleElement45,TupleElement46,TupleElement47,TupleElement48,TupleElement49,TupleElement50,TupleElement51,TupleElement52,TupleElement53,TupleElement54) as frame_support::traits::hooks::OnFinalize<BlockNumber>>::on_finalize::h185b0edbdb72401e 4: 0x2e981b - pendulum_runtime.wasm!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::idle_and_finalize_hook::h923d32de48ac8de7 5: 0x3e02c3 - pendulum_runtime.wasm!BlockBuilder_finalize_block } 2025-08-04 10:06:49 [Parachain] 💤 Idle (15 peers), best: #5626322 (0x1d5e…20fd), finalized #5626320 (0x005f…15ac), ⬇ 18.0kiB/s ⬆ 1.2kiB/s 2025-08-04 10:06:50 [Relaychain] 🔍 Discovered new external address for our node: /ip4/164.100.10.1/tcp/30366/ws/p2p/12D3KooWNTuLVhp7zgUK19TLrSmcrp7mRDqNUvDvQv5HSxok5Dhp

Upgrade1.6.0-a -> 1.6.0-b

2025-08-04 10:11:12 [Parachain] Reward per block is None 2025-08-04 10:11:12 [Parachain] panicked at /home/builder/cargo/git/checkouts/polkadot-sdk-2a8e7ffb960df38b/0a31531/cumulus/pallets/parachain-system/src/lib.rs:1339:3: no space left for the block in the unincluded segment 2025-08-04 10:11:12 [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back. 2025-08-04 10:11:12 [Parachain] 1 storage transactions are left open by the runtime. Those will be rolled back. 2025-08-04 10:11:12 [Parachain] ❗️ Inherent extrinsic returned unexpected error: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm unreachable instruction executed WASM backtrace: error while executing at wasm backtrace: 0: 0x75b494 - pendulum_runtime.wasm!rust_begin_unwind 1: 0xc93a - pendulum_runtime.wasm!core::panicking::panic_fmt::hac62c46e7ac26bea 2: 0x3d4412 - pendulum_runtime.wasm!cumulus_pallet_parachain_system::<impl cumulus_pallet_parachain_system::pallet::Pallet<T>>::maybe_drop_included_ancestors::h0ca6ac067ab071b1 3: 0x66f91d - pendulum_runtime.wasm!frame_support::storage::transactional::with_transaction::h7056de9b958d3a12 4: 0x2b27b6 - pendulum_runtime.wasm!<cumulus_pallet_parachain_system::pallet::Call<T> as frame_support::traits::dispatch::UnfilteredDispatchable>::dispatch_bypass_filter::{{closure}}::heb5b0ab65d3355fd 5: 0x2b8105 - pendulum_runtime.wasm!environmental::local_key::LocalKey<T>::with::h55450cbda69d64e5 6: 0x9d1db - pendulum_runtime.wasm!<pendulum_runtime::RuntimeCall as frame_support::traits::dispatch::UnfilteredDispatchable>::dispatch_bypass_filter::h2e8ebd3a47e1e452 7: 0x6d3036 - pendulum_runtime.wasm!<sp_runtime::generic::checked_extrinsic::CheckedExtrinsic<AccountId,Call,Extra> as sp_runtime::traits::Applyable>::apply::h8aff7ef5e9718e49 8: 0x2e93a6 - pendulum_runtime.wasm!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::apply_extrinsic::h5a50b7fbfdc60171 9: 0x3e0167 - pendulum_runtime.wasm!BlockBuilder_apply_extrinsic. Dropping. 2025-08-04 10:11:12 [Parachain] panicked at /home/builder/cargo/git/checkouts/polkadot-sdk-2a8e7ffb960df38b/0a31531/cumulus/pallets/parachain-system/src/lib.rs:265:6: set_validation_data inherent needs to be present in every block! 2025-08-04 10:11:12 [Parachain] err=Error { inner: Proposing Caused by: 0: Error at calling runtime api: Execution failed: Execution aborted due to trap: wasm trap: wasm unreachable instruction executed WASM backtrace: error while executing at wasm backtrace: 0: 0x75b494 - pendulum_runtime.wasm!rust_begin_unwind 1: 0xc93a - pendulum_runtime.wasm!core::panicking::panic_fmt::hac62c46e7ac26bea 2: 0xfe9c - pendulum_runtime.wasm!core::option::expect_failed::hc18f070396e13242 3: 0x379a19 - pendulum_runtime.wasm!<(TupleElement0,TupleElement1,TupleElement2,TupleElement3,TupleElement4,TupleElement5,TupleElement6,TupleElement7,TupleElement8,TupleElement9,TupleElement10,TupleElement11,TupleElement12,TupleElement13,TupleElement14,TupleElement15,TupleElement16,TupleElement17,TupleElement18,TupleElement19,TupleElement20,TupleElement21,TupleElement22,TupleElement23,TupleElement24,TupleElement25,TupleElement26,TupleElement27,TupleElement28,TupleElement29,TupleElement30,TupleElement31,TupleElement32,TupleElement33,TupleElement34,TupleElement35,TupleElement36,TupleElement37,TupleElement38,TupleElement39,TupleElement40,TupleElement41,TupleElement42,TupleElement43,TupleElement44,TupleElement45,TupleElement46,TupleElement47,TupleElement48,TupleElement49,TupleElement50,TupleElement51,TupleElement52,TupleElement53,TupleElement54) as frame_support::traits::hooks::OnFinalize<BlockNumber>>::on_finalize::h185b0edbdb72401e 4: 0x2e981b - pendulum_runtime.wasm!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::idle_and_finalize_hook::h923d32de48ac8de7 5: 0x3e02c3 - pendulum_runtime.wasm!BlockBuilder_finalize_block 1: Execution failed: Execution aborted due to trap: wasm trap: wasm unreachable instruction executed WASM backtrace: error while executing at wasm backtrace: 0: 0x75b494 - pendulum_runtime.wasm!rust_begin_unwind 1: 0xc93a - pendulum_runtime.wasm!core::panicking::panic_fmt::hac62c46e7ac26bea 2: 0xfe9c - pendulum_runtime.wasm!core::option::expect_failed::hc18f070396e13242 3: 0x379a19 - pendulum_runtime.wasm!<(TupleElement0,TupleElement1,TupleElement2,TupleElement3,TupleElement4,TupleElement5,TupleElement6,TupleElement7,TupleElement8,TupleElement9,TupleElement10,TupleElement11,TupleElement12,TupleElement13,TupleElement14,TupleElement15,TupleElement16,TupleElement17,TupleElement18,TupleElement19,TupleElement20,TupleElement21,TupleElement22,TupleElement23,TupleElement24,TupleElement25,TupleElement26,TupleElement27,TupleElement28,TupleElement29,TupleElement30,TupleElement31,TupleElement32,TupleElement33,TupleElement34,TupleElement35,TupleElement36,TupleElement37,TupleElement38,TupleElement39,TupleElement40,TupleElement41,TupleElement42,TupleElement43,TupleElement44,TupleElement45,TupleElement46,TupleElement47,TupleElement48,TupleElement49,TupleElement50,TupleElement51,TupleElement52,TupleElement53,TupleElement54) as frame_support::traits::hooks::OnFinalize<BlockNumber>>::on_finalize::h185b0edbdb72401e 4: 0x2e981b - pendulum_runtime.wasm!frame_executive::Executive<System,Block,Context,UnsignedValidator,AllPalletsWithSystem,COnRuntimeUpgrade>::idle_and_finalize_hook::h923d32de48ac8de7 5: 0x3e02c3 - pendulum_runtime.wasm!BlockBuilder_finalize_block } 2025-08-04 10:11:13 [Parachain] 💤 Idle (15 peers), best: #5626338 (0x52cf…6623), finalized #5626336 (0xfa7f…194a), ⬇ 0.6kiB/s ⬆ 0.6kiB/s

Now running 1.6.0-b which seems to produce blocks despite of the errors

Aug 04 '25 10:08 stakeworks

Hmm. Did you wipe your relaychain and parachain databases before syncing? You can make them catch up quickly with the --sync warp flag. We are running the 1.6.0-d version in production and they don't throw the errors you shared. What flags are you using when running your node?

Aug 04 '25 12:08 ebma

Can you accept friend request on Discord, I want to discuss my collators first

Aug 05 '25 09:08 stakeworks

Been a busy week, but both our amplitude and pendulim nodes are syncing with v1.6.0-d. Have not had any issues so far

Aug 08 '25 14:08 gflow33