roller
roller copied to clipboard
failed to retrieve batch from DA
my rollapp height stop at 2777 while the Hub at 2778 and it's maybe caursed by sudden server down because of celestia light node
tail -f rollapp.log
shows
level=error msg="failed to retrieve batch from DA[daHeight 463809 error failed to retrieve batch: Get \"http://localhost:26659/namespaced_data/0000000000000000000000000000000000000000000dd20849c39df9d8/height/463809\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)]" module=BlockManager
level=error msg="Failed to sync until target. error while processing next DA batch[error failed to retrieve batch: Get \"http://localhost:26659/namespaced_data/0000000000000000000000000000000000000000000dd20849c39df9d8/height/463809\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)]" module=BlockManager
after restart the sequencer service, the daHeight in level=debug msg="trying to retrieve batch from DA[daHeight 463809]" module=BlockManager
may increase few hundred but finally show failed to retrieve batch from DA
my celestia node is synced and don't changed da layer, also try sync from snapshot but the snapshot at height 2777, so nothing seems to have changed. sometimes the link above will need more time to access or back the results in the comment below, and seems it won't retry to access the link after failing once and need restart the service manually.
and i want to ask if there are anywhy can rollback the Hub height
sometimes access the link it will show
"protocols not supported: [/arabica-9/shrex/nd/v0.0.2]\ncontext deadline exceeded\ncontext deadline exceeded\ngetter/ipld: failed to retrieve shares by namespace: retrieving shares by namespace 0000000000000000000000000000000000000000000dd20849c39df9d8 for row 626167616f34616d6235796176756161616161616161616161616161616161616161616161616161616161616161616161616161616161616161616361616161616161616161616161616161616161616161616161616161616161616161736b6873776275623570326977627433746b6c6f73357579736370746236796965656f7970696e637179716376626b7a75766e35683376327a79726478696f77: context deadline exceeded"
Duplicate of #571
there is an known outage in celestia devnet network currently
I don't think it's related as #571 is two weeks old @omritoptix
@DreamGallery are you running the rollapp node in an aggregator=false mode? if so can you explain why? Seems to me like the issue is that you can't get the batch from the DA (which contains the latest blocks) hence you can't change mode to aggregator=true (i.e becoming a sequencer) but I need to get more details to understand the cause for needing to sync from the DA.
In discord, I saw that the #571 seems not be solved and he create a new rollapp @ItayLevyOfficial
@omritoptix
I don‘t know why maybe an unexpected down of vps because of celestia light node that cause my node height stuck at 2777 but the Hub height at 2778 after restart. So I don't know why my local height will be one height behind the Hub
I tried many ways and of course also saw the #571, so I tried the way that @ItayLevyOfficial mentioned in #571
https://docs.dymension.xyz/build/state_sync
sync node in aggregator=false mode
from snapshot or from the 0 height
if the aggregator=true
, this is the results(also the fisrt restart after vps down I saw in log)
in #571, the man want to move his node to another vps, but for me, I just saw my node in https://portal.dymension.xyz/rollapp/gallery_1047877-1 become red so I restart the rollapp service And I also met a vps stuck during these days that I restart the VPS. The service restart is behind the VPS dwon, and I just restart the service and thought that this would be normal after restart so I did't see the log after restart and nothing changed after my first service restart at Nov 2, so the problem maybe happened before the VPS down.