reth icon indicating copy to clipboard operation
reth copied to clipboard

feat(cli): override static files datadir

Open justcode740 opened this issue 1 year ago • 2 comments

busy af, so not have time on this for a bit but beside some doc , should work now, rely on review for nits, if u can help solve these great o.w. have to come back wen have time.. resolves #6945

@shekhirin could you review it? @gakonst this guy is busy af.

justcode740 avatar Mar 19 '24 06:03 justcode740

Hey @justcode740! Let me know if we can help get this over the line 😊

onbjerg avatar Apr 18 '24 18:04 onbjerg

Works as intended

➜  reth git:(0x/override_static_datadir) cargo run -- db --chain holesky --datadir ~/Library/Application\ Support/reth/holesky  stats                                                                                 
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.43s
     Running `target/debug/reth db --chain holesky --datadir '/Users/shekhirin/Library/Application Support/reth/holesky' stats`
| Segment | Block Range | Transaction Range | Shape (columns x rows) | Size |
|---------|-------------|-------------------|------------------------|------|
| ------- | ----------- | ----------------- | ---------------------- | ---- |
| Total   |             |                   |                        | 0 B  |


| Table Name                 | # Entries | Branch Pages | Leaf Pages | Overflow Pages | Total Size |
|----------------------------|-----------|--------------|------------|----------------|------------|
| AccountChangeSets          | 69674674  | 149552       | 813634     | 0              | 3.7 GiB    |
| AccountsHistory            | 23464603  | 8912         | 627740     | 4575           | 2.4 GiB    |
| AccountsTrie               | 3071552   | 1769         | 236758     | 0              | 931.7 MiB  |
| BlockBodyIndices           | 1509089   | 41           | 8831       | 0              | 34.7 MiB   |
| BlockOmmers                | 0         | 0            | 0          | 0              | 0 B        |
| BlockWithdrawals           | 1499143   | 958          | 214106     | 0              | 840.1 MiB  |
| Bytecodes                  | 80842     | 69           | 4786       | 166114         | 667.8 MiB  |
| CanonicalHeaders           | 29        | 1            | 2          | 0              | 12 KiB     |
| HashedAccounts             | 38463002  | 17013        | 1077033    | 0              | 4.2 GiB    |
| HashedStorages             | 73357766  | 24389        | 1416292    | 0              | 5.5 GiB    |
| HeaderNumbers              | 1509088   | 506          | 32969      | 0              | 130.8 MiB  |
| HeaderTerminalDifficulties | 29        | 0            | 1          | 0              | 4 KiB      |
| Headers                    | 29        | 1            | 6          | 0              | 28 KiB     |
| PlainAccountState          | 38463002  | 8094         | 735703     | 0              | 2.8 GiB    |
| PlainStorageState          | 73357766  | 26534        | 1344640    | 0              | 5.2 GiB    |
| PruneCheckpoints           | 4         | 0            | 1          | 0              | 4 KiB      |
| Receipts                   | 1228      | 3            | 61         | 40             | 416 KiB    |
| StageCheckpointProgresses  | 1         | 0            | 1          | 0              | 4 KiB      |
| StageCheckpoints           | 12        | 0            | 1          | 0              | 4 KiB      |
| StorageChangeSets          | 99867933  | 206829       | 1517009    | 0              | 6.6 GiB    |
| StoragesHistory            | 29066521  | 27328        | 1177160    | 2188           | 4.6 GiB    |
| StoragesTrie               | 6079323   | 52025        | 420964     | 0              | 1.8 GiB    |
| TransactionBlocks          | 1333963   | 40           | 8552       | 0              | 33.6 MiB   |
| TransactionHashNumbers     | 51244887  | 10886        | 862829     | 0              | 3.3 GiB    |
| TransactionSenders         | 33473229  | 1399         | 312835     | 0              | 1.2 GiB    |
| Transactions               | 1228      | 4            | 91         | 294            | 1.5 MiB    |
| VersionHistory             | 4         | 0            | 1          | 0              | 4 KiB      |
| -------------------------- | --------- | ------------ | ---------- | -------------- | ---------- |
| Tables                     |           |              |            |                | 44 GiB     |
| Freelist                   | 426014    |              |            |                | 1.6 GiB    |

➜  reth git:(0x/override_static_datadir) cargo run -- db --chain holesky --datadir ~/Library/Application\ Support/reth/holesky --datadir.static_files ~/Library/Application\ Support/reth/holesky/custom_static_files_dir s
tats
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.42s
     Running `target/debug/reth db --chain holesky --datadir '/Users/shekhirin/Library/Application Support/reth/holesky' --datadir.static_files '/Users/shekhirin/Library/Application Support/reth/holesky/custom_static_fi
les_dir' stats`
| Segment      | Block Range | Transaction Range | Shape (columns x rows) | Size      |
|--------------|-------------|-------------------|------------------------|-----------|
| Headers      | 0..=1509088 | N/A               | 3 x 1509089            | 705.3 MiB |
| Transactions | 0..=1509088 | 0..=51244886      | 1 x 51244887           | 37 GiB    |
| Receipts     | 0..=1509088 | 0..=51244886      | 1 x 51244887           | 5.8 GiB   |
| ------------ | ----------- | ----------------- | ---------------------- | --------- |
| Total        |             |                   |                        | 43.5 GiB  |


| Table Name                 | # Entries | Branch Pages | Leaf Pages | Overflow Pages | Total Size |
|----------------------------|-----------|--------------|------------|----------------|------------|
| AccountChangeSets          | 69674674  | 149552       | 813634     | 0              | 3.7 GiB    |
| AccountsHistory            | 23464603  | 8912         | 627740     | 4575           | 2.4 GiB    |
| AccountsTrie               | 3071552   | 1769         | 236758     | 0              | 931.7 MiB  |
| BlockBodyIndices           | 1509089   | 41           | 8831       | 0              | 34.7 MiB   |
| BlockOmmers                | 0         | 0            | 0          | 0              | 0 B        |
| BlockWithdrawals           | 1499143   | 958          | 214106     | 0              | 840.1 MiB  |
| Bytecodes                  | 80842     | 69           | 4786       | 166114         | 667.8 MiB  |
| CanonicalHeaders           | 29        | 1            | 2          | 0              | 12 KiB     |
| HashedAccounts             | 38463002  | 17013        | 1077033    | 0              | 4.2 GiB    |
| HashedStorages             | 73357766  | 24389        | 1416292    | 0              | 5.5 GiB    |
| HeaderNumbers              | 1509088   | 506          | 32969      | 0              | 130.8 MiB  |
| HeaderTerminalDifficulties | 29        | 0            | 1          | 0              | 4 KiB      |
| Headers                    | 29        | 1            | 6          | 0              | 28 KiB     |
| PlainAccountState          | 38463002  | 8094         | 735703     | 0              | 2.8 GiB    |
| PlainStorageState          | 73357766  | 26534        | 1344640    | 0              | 5.2 GiB    |
| PruneCheckpoints           | 4         | 0            | 1          | 0              | 4 KiB      |
| Receipts                   | 1228      | 3            | 61         | 40             | 416 KiB    |
| StageCheckpointProgresses  | 1         | 0            | 1          | 0              | 4 KiB      |
| StageCheckpoints           | 12        | 0            | 1          | 0              | 4 KiB      |
| StorageChangeSets          | 99867933  | 206829       | 1517009    | 0              | 6.6 GiB    |
| StoragesHistory            | 29066521  | 27328        | 1177160    | 2188           | 4.6 GiB    |
| StoragesTrie               | 6079323   | 52025        | 420964     | 0              | 1.8 GiB    |
| TransactionBlocks          | 1333963   | 40           | 8552       | 0              | 33.6 MiB   |
| TransactionHashNumbers     | 51244887  | 10886        | 862829     | 0              | 3.3 GiB    |
| TransactionSenders         | 33473229  | 1399         | 312835     | 0              | 1.2 GiB    |
| Transactions               | 1228      | 4            | 91         | 294            | 1.5 MiB    |
| VersionHistory             | 4         | 0            | 1          | 0              | 4 KiB      |
| -------------------------- | --------- | ------------ | ---------- | -------------- | ---------- |
| Tables                     |           |              |            |                | 44 GiB     |
| Freelist                   | 426014    |              |            |                | 1.6 GiB    |

shekhirin avatar May 16 '24 10:05 shekhirin

yeah sg, either way is fine with me

joshieDo avatar Jun 03 '24 16:06 joshieDo