reth icon indicating copy to clipboard operation
reth copied to clipboard

Prune config is no effect and receipts data exists in db tables

Open sysvm opened this issue 1 year ago • 2 comments

Describe the bug

I met a problem when running reth with prune config

  1. run reth in archive node without prune config, and set --debug.max-block=7000000, when ending import blocks, using db stats to analyse db, there are receipts data in static file: 22.3Gib and receipts in db is no data

  2. run reth in archive node with prune config, and set --debug.max-block=7000000, when ending import blocks, using db stats to analyse db, there are no receipts data in static file, there is receipts data in db: 25.3 GiB. Config as follows:

[prune]
block_interval = 5
[prune.parts]
receipts = {  distance = 3_000_000}
sender_recovery = { distance = 3_000_000 }
account_history = { distance = 3_000_000 }
storage_history = { distance = 3_000_000}
transaction_lookup = {distance = 3_000_000}

So why I use prune config in reth.toml and prune receipt, but the result is no effect and the receipts data is in db

Steps to reproduce

  1. Configure the reth.toml with prune config as follows:
[prune]
block_interval = 5
[prune.parts]
receipts = {  distance = 3_000_000}
sender_recovery = { distance = 3_000_000 }
account_history = { distance = 3_000_000 }
storage_history = { distance = 3_000_000}
transaction_lookup = {distance = 3_000_000}
  1. Start running the node with CLI: nohup ./reth node --datadir ./data --authrpc.jwtsecret /server/test/reth/jwt.hex --authrpc.addr 127.0.0.1 --authrpc.port 8551 --http --http.port 9010 --http.api all --debug.max-block 5000000 --log.file.directory ./logs --metrics 0.0.0.0:6060 >> nohup.out 2>&1&
  2. When ending importing blocks, use ./reth db --datadir ./data/ stats to analyze the db, the receipt data is shown in db table image
  3. But when I run reth without ant prune config, use ./reth db --datadir ./data/ stats to analyze the db, the receipts data is shown in static file: image

Node logs

INFO Pruner initialized prune_config=PruneConfig { block_interval: 5, segments: PruneModes { sender_recovery: Some(Distance(3000000)), transaction_lookup: Some(Distance(3000000)), receipts: Some(Distance(3000000)), account_history: Some(Distance(3000000)), storage_history: Some(Distance(3000000)), receipts_log_filter: ReceiptsLogPruneConfig({}) } }

Platform(s)

Linux (x86)

What version/commit are you on?

Reth 1.0.3

What database version are you on?

Current database version: 2 Local database version: 2

Which chain / network are you on?

default , mainnet of eth

What type of node are you running?

Archive (default)

What prune config do you use, if any?

[prune]
block_interval = 5
[prune.parts]
receipts = {  distance = 3_000_000}
sender_recovery = { distance = 3_000_000 }
account_history = { distance = 3_000_000 }
storage_history = { distance = 3_000_000}
transaction_lookup = {distance = 3_000_000}

If you've built Reth from source, provide the full command you used

No response

Code of Conduct

  • [X] I agree to follow the Code of Conduct

sysvm avatar Aug 06 '24 06:08 sysvm

Hello @sysvm please can I be assigned to this issue?

ShantelPeters avatar Aug 06 '24 08:08 ShantelPeters

Hello @sysvm please can I be assigned to this issue?

I have no permission to assign it.

sysvm avatar Aug 06 '24 10:08 sysvm

This issue is stale because it has been open for 21 days with no activity.

github-actions[bot] avatar Aug 28 '24 01:08 github-actions[bot]

This issue is stale because it has been open for 21 days with no activity.

github-actions[bot] avatar Sep 20 '24 02:09 github-actions[bot]

could it be that https://github.com/paradigmxyz/reth/pull/10639 and https://github.com/paradigmxyz/reth/pull/10774 have solved this issue @sysvm ?

emhane avatar Sep 23 '24 14:09 emhane

This issue is stale because it has been open for 21 days with no activity.

github-actions[bot] avatar Oct 18 '24 02:10 github-actions[bot]

This issue was closed because it has been inactive for 7 days since being marked as stale.

github-actions[bot] avatar Oct 25 '24 02:10 github-actions[bot]

it seems by design, see the following code, crates/static-file/static-file/src/static_file_producer.rs

    /// Returns a static file targets at the provided finalized block numbers per segment.
    /// The target is determined by the check against highest `static_files` using
    /// [`reth_provider::providers::StaticFileProvider::get_highest_static_files`].
    pub fn get_static_file_targets(
        &self,
        finalized_block_numbers: HighestStaticFiles,
    ) -> ProviderResult<StaticFileTargets> {
        let highest_static_files = self.provider.static_file_provider().get_highest_static_files();

        let targets = StaticFileTargets {
            headers: finalized_block_numbers.headers.and_then(|finalized_block_number| {
                self.get_static_file_target(highest_static_files.headers, finalized_block_number)
            }),
            // StaticFile receipts only if they're not pruned according to the user configuration
            receipts: if self.prune_modes.receipts.is_none() &&
                self.prune_modes.receipts_log_filter.is_empty()
            {
                finalized_block_numbers.receipts.and_then(|finalized_block_number| {
                    self.get_static_file_target(
                        highest_static_files.receipts,
                        finalized_block_number,
                    )
                })
            } else {
                None
            },
            transactions: finalized_block_numbers.transactions.and_then(|finalized_block_number| {
                self.get_static_file_target(
                    highest_static_files.transactions,
                    finalized_block_number,
                )
            }),
        };

        trace!(
            target: "static_file",
            ?finalized_block_numbers,
            ?highest_static_files,
            ?targets,
            any = %targets.any(),
            "StaticFile targets"
        );

        Ok(targets)
    }

if any prune config is set, we will get a none receipts target:

            receipts: if self.prune_modes.receipts.is_none() &&
                self.prune_modes.receipts_log_filter.is_empty()
            {
                finalized_block_numbers.receipts.and_then(|finalized_block_number| {
                    self.get_static_file_target(
                        highest_static_files.receipts,
                        finalized_block_number,
                    )
                })
            } else {
                None
            },

Is my understanding correct?

DevinByte avatar Jan 24 '25 12:01 DevinByte