ckb icon indicating copy to clipboard operation
ckb copied to clipboard

[BREAKING CHANGE] Hardcoding a Default `assume_valid_target` to Reduce the Timecost of Block Synchronization in IBD mode

Open eval-exec opened this issue 2 years ago • 3 comments

What problem does this PR solve?

Issue Number: close #4253

What's Changed:

Related changes

  • Set a block in 60 days ago as default assume_valid_target if user doesn't explictly specify --assume-valid-target argument.
  • User specified --assume-valid-target will overwrite the default hardcoded assume_valid_target
  • If user specify --assume-valid-target 0x0000000000000000000000000000000000000000000000000000000000000000, ckb will do full verification.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code ci-runs-only: [ quick_checks,linters ]

Side effects

  • None

Release note

Title Only: Include only the PR title in the release note.

eval-exec avatar Dec 04 '23 09:12 eval-exec

In the early phase of "finding block header of assume_valid_target" CKB will print many logs like:

32(0xe9b97767424dd04aa65a1f7ad562b0faf8dd0fbf2a213d1586ea7969160f5996) please wait
2023-12-04 07:51:20.256 +00:00 BlockDownload INFO ckb_sync::synchronizer  best known header number: 10890000, hash: Byte32(0x662ba941ac8dd506832704a35fc13534e1ac402aa708670936af03d27114c643), can't find assume valid target temporarily, hash: Byte32(0xe9b97767424dd04aa65a1f7ad562b0faf8dd0fbf2a213d1586ea7969160f5996) please wait
2023-12-04 07:51:22.676 +00:00 BlockDownload INFO ckb_sync::synchronizer  best known header number: 10900000, hash: Byte32(0xc793a8184a272be18a0736b4a3713f19f75ba9f8913c88ca8d65b05a03304306), can't find assume valid target temporarily, hash: Byte32(0xe9b97767424dd04aa65a1f7ad562b0faf8dd0fbf2a213d1586ea7969160f5996) please wait
2023-12-04 07:51:25.162 +00:00 BlockDownload INFO ckb_sync::synchronizer  best known header number: 10910000, hash: Byte32(0xa7d0e834174d4c0677648c66b8d74868ce4dcf8514d8f7c40c909cf6f68578ff), can't find assume valid target temporarily, hash: Byte32(0xe9b97767424dd04aa65a1f7ad562b0faf8dd0fbf2a213d1586ea7969160f5996) please wait
2023-12-04 07:51:27.096 +00:00 BlockDownload INFO ckb_sync::synchronizer  best known header number: 10920000, hash: Byte32(0xe4162e3db5b73eedffa5267595f4b6596cf765b5b6afeae7b1e1649eff057eed), can't find assume valid target temporarily, hash: Byte32(0xe9b97767424dd04aa65a1f7ad562b0faf8dd0fbf2a213d1586ea7969160f5996) please wait
2023-12-04 07:51:29.217 +00:00 BlockDownload INFO ckb_sync::synchronizer  best known header number: 10930000, hash: Byte32(0x43ce8a7e8c1d65f313d8d1b217145477d5cbc24e12a5ffc9104da2e7ab0d0a04), can't find assume valid target temporarily, hash: Byte32(0xe9b97767424dd04aa65a1f7ad562b0faf8dd0fbf2a213d1586ea7969160f5996) please wait
2023-12-04 07:51:31.193 +00:00 BlockDownload INFO ckb_sync::synchronizer  best known header number: 10940000, hash: Byte32(0x418fe666fa6c47b98f79c2f6d579b590e4c3449dc555614063f4d3450df32852), can't find assume valid target temporarily, hash: Byte32(0xe9b97767424dd04aa65a1f7ad562b0faf8dd0fbf2a213d1586ea7969160f5996) please wait
2023-12-04 07:51:33.148 +00:00 BlockDownload INFO ckb_sync::synchronizer  best known header number: 10950000, hash: Byte32(0xa17857353561c5488bb0d82f4ad60114c96a3032eb9cbca771fc40e43ecb51c0), can't find assume valid target temporarily, hash: Byte32(0xe9b97767424dd04aa65a1f7ad562b0faf8dd0fbf2a213d1586ea7969160f5996) please wait

These logs will appear about 1 hour, need to reword these into more friendly message, like: INFO current best known header: {number}-{hash}, assume_valid_target: {number}-{hash}, CKB need {duration} to reach the target, progress {27%}

eval-exec avatar Dec 05 '23 08:12 eval-exec

@doitian How about creating a template file named default_assume_valid_target.rs.template and then using envsubst to generate default_assume_valid_target.rs from this template file?

Ref: https://www.baeldung.com/linux/envsubst-command

eval-exec avatar Mar 22 '24 08:03 eval-exec

@doitian How about creating a template file named default_assume_valid_target.rs.template and then using envsubst to generate default_assume_valid_target.rs from this template file?

Ref: https://www.baeldung.com/linux/envsubst-command

I think you can just use shell HEREDOC string

cat <<EOF
/// The \`mod mainnet\` and \`mod testnet\`'s codes are generated
/// by script: ./devtools/release/update_default_valid_target.sh

/// sync config related to mainnet
pub mod mainnet {
    // Default assume valid target for mainnet, expect to be a block 60 days ago.
    // Need to update when CKB's new release
    // in mainnet: the 60 days ago block is:
    // height: $MAINNET_HEIGHT
    // hash: $MAINNET_HASH
    // date: $GENERATED_AT
    // you can view this block in https://explorer.nervos.org/block/$MAINNET_HASH
    pub const DEFAULT_ASSUME_VALID_TARGET: &str =
        "$MAINNET_HASH";
}

/// sync config related to testnet
pub mod testnet {
    // Default assume valid target for testnet, expect to be a block 60 days ago.
    // Need to update when CKB's new release
    // in testnet: the 60 days ago block is:
    // height: $TESTNET_HEIGHT
    // hash: $TESTNET_HASH
    // date: $GENERATED_AT
    // you can view this block in https://pudge.explorer.nervos.org/block/$TESTNET_HASH
    pub const DEFAULT_ASSUME_VALID_TARGET: &str =
        "$TESTNET_HASH";
}
EOF

doitian avatar Mar 22 '24 08:03 doitian

missing documentation for a module

doitian avatar Mar 27 '24 01:03 doitian

https://stackoverflow.com/questions/1167746/how-to-assign-a-heredoc-value-to-a-variable-in-bash

eval-exec avatar Mar 27 '24 08:03 eval-exec

I will upload the comparison chart of the full block synchronization results between this PR and v0.115.0-rc2 in a few hours.

eval-exec avatar Mar 27 '24 09:03 eval-exec

I will upload the comparison chart of the full block synchronization results between this PR and v0.115.0-rc2 in a few hours.

The comparison chart has been uploaded to the PR's description.

I think this PR is ready to be reviewed.

eval-exec avatar Mar 28 '24 02:03 eval-exec