celestia-node
celestia-node copied to clipboard
celestiaorg
Providing trusted-hash produces daser errors
Celestia Node version
a7c49d20062327b636e7e08d859802a8f5393ac0
OS
linux/amd64 and macos/arm64
Install tools
docker or vanilla install
Others
No response
Steps to reproduce it
- Initialise a new da-light node
- Execute this command
celestia light start --p2p.network mocha --core.ip http://consensus-full-mocha-4.celestia-mocha.com --headers.trusted-hash B1930EF8E01DE5684D854ACE8730C16F4498CE3BB2E2EE1C2000CF7FA3F7F875
Expected result
node starts at a non-genesis height without errors
Actual result
daser errors with failed to get header from header store
Relevant log output
2024-02-05T13:27:03.622+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 116924, "to": 116924, "errors": 1, "finished (s)": 0.000490375}
2024-02-05T13:27:03.623+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 101613, "finished (s)": 0.000004917}
2024-02-05T13:27:03.621+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 66038, "finished (s)": 0.000006417}
2024-02-05T13:27:03.622+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 86816, "finished (s)": 0.000011208}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 86816, "to": 86816, "errors": 1, "finished (s)": 0.000419916}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 101613, "to": 101613, "errors": 1, "finished (s)": 0.0000535}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 66038, "to": 66038, "errors": 1, "finished (s)": 0.001750667}
2024-02-05T13:27:03.623+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 31754, "finished (s)": 0.00000625}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 31754, "to": 31754, "errors": 1, "finished (s)": 0.000036833}
2024-02-05T13:27:03.623+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 46092, "finished (s)": 0.00000475}
2024-02-05T13:27:03.623+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 47513, "finished (s)": 0.000006}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 47513, "to": 47513, "errors": 1, "finished (s)": 0.000013}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 46092, "to": 46092, "errors": 1, "finished (s)": 0.000037667}
2024-02-05T13:27:03.623+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 24177, "finished (s)": 0.000003875}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 24177, "to": 24177, "errors": 1, "finished (s)": 0.00002225}
2024-02-05T13:27:03.623+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 107287, "finished (s)": 0.000007833}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 107287, "to": 107287, "errors": 1, "finished (s)": 0.000381417}
2024-02-05T13:27:03.623+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 51372, "finished (s)": 0.000005125}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 51372, "to": 51372, "errors": 1, "finished (s)": 0.000033208}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 71673, "to": 71673, "errors": 1, "finished (s)": 0.002123125}
2024-02-05T13:27:03.622+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 16583, "finished (s)": 0.00000475}
2024-02-05T13:27:03.623+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 39254, "finished (s)": 0.000006125}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 16583, "to": 16583, "errors": 1, "finished (s)": 0.000983458}
2024-02-05T13:27:03.623+0100 INFO das das/worker.go:88 finished sampling headers {"type": "retry", "from": 39254, "to": 39254, "errors": 1, "finished (s)": 0.000012208}
Notes
I also see that the daser can fail the whole 100 batches instead of just 1 like in the shared log
Same error with latest commit 2ebca8f6b96da47f50f4844b98e503b8cd2d9aeb @Wondertan I changed the trusted hash from the flag to the config.toml
celestia light start --p2p.network mocha --core.ip http://consensus-full-mocha-4.celestia-mocha.com
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121267, "finished (s)": 0.000005041}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121268, "finished (s)": 0.000000791}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121269, "finished (s)": 0.000000833}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121270, "finished (s)": 0.000000792}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121271, "finished (s)": 0.00000175}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121272, "finished (s)": 0.000016125}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121273, "finished (s)": 0.000003042}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121274, "finished (s)": 0.000001083}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121275, "finished (s)": 0.000000666}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121276, "finished (s)": 0.000000708}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121277, "finished (s)": 0.000001666}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121278, "finished (s)": 0.000000667}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121279, "finished (s)": 0.000001125}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121280, "finished (s)": 0.000004958}
2024-02-27T14:35:29.927+0100 ERROR das das/worker.go:170 failed to get header from header store {"height": 121281, "finished (s)": 0.000000959}
A little guide to running a celestia node on a public network without waiting for it to sync:
- Init a fresh node
- Change the two following config fields for the recent header(the example is from mocha):
- Header.TrustedHash = "0E005E02A1EE6F9350E2B74EF388925F65B8EC1E6D11E3DD92DDD20C82A860F8"
- DASer.SampleFrom = 1213740
- Start the node
A little guide to running a celestia node on a public network without waiting for it to sync:
Init a fresh node
Change the two following config fields for the recent header(the example is from mocha):
- Header.TrustedHash = "0E005E02A1EE6F9350E2B74EF388925F65B8EC1E6D11E3DD92DDD20C82A860F8"
- DASer.SampleFrom = 1213740
Start the node
Works! Thanks for the review. for broader context - should we treat this as a bug or CR (Change Request) @ramin ? As there is no vanilla way through cli flags to provide daser.SampleFrom
The cli flag can help immensely devRel and external teams devX wise