nethermind icon indicating copy to clipboard operation
nethermind copied to clipboard

Added more use of ProgressLogger (#8504)

Open Vivkzz opened this issue 9 months ago • 7 comments

Fixes Closes Resolves #8504

Changes:

  • Added ProgressLogger to the FullPruningDb class to show progress while pruning.
  • The logger starts when pruning begins and updates as keys are processed.
  • Cleaned up the logger after the pruning is done to keep things tidy.

Types of changes

What types of changes does your code introduce?

  • [X] Optimization
  • [X] Refactoring
  • [X] Documentation update
  • [X] Build-related changes

Testing

Requires testing

  • [] Yes
  • [X] No

If yes, did you write tests?

  • [ ] Yes
  • [ ] No

Notes on testing

tested this locally, and the ProgressLogger works great, showing progress during the pruning process.

Documentation

Requires documentation update

  • [ ] Yes
  • [X] No

If yes, link the PR to the docs update or the issue with the details labeled docs. Remove if not applicable.

Requires explanation in Release Notes

  • [ ] Yes
  • [ ] No

If yes, fill in the details here. Remove if not applicable.

Remarks

This change makes it easier to see what's happening during the pruning process, which is super helpful for long-running tasks.

Vivkzz avatar Apr 10 '25 11:04 Vivkzz

Did you run it?

asdacap avatar Apr 10 '25 11:04 asdacap

As in, run full pruning.

asdacap avatar Apr 10 '25 11:04 asdacap

@LukaszRozmej @asdacap Thanks for the feedback! I've made the following changes:

  • Increased the progress logging threshold to 100000 keys as suggested
  • Removed the TotalKeys calculation to avoid incorrect upper bound
  • Progress logger now tracks actual processed keys

Regarding the question about running full pruning - I haven't run it yet.

Vivkzz avatar Apr 11 '25 08:04 Vivkzz

@LukaszRozmej @asdacap Thanks for the feedback! I've made the following changes:

  • Increased the progress logging threshold to 100000 keys as suggested
  • Removed the TotalKeys calculation to avoid incorrect upper bound
  • Progress logger now tracks actual processed keys

Regarding the question about running full pruning - I haven't run it yet.

Forgot to push? Please run and test and share new logs.

LukaszRozmej avatar Apr 11 '25 10:04 LukaszRozmej

@LukaszRozmej i already pushed it are u able to see it ?

Vivkzz avatar Apr 11 '25 11:04 Vivkzz

@LukaszRozmej kindly check and update me

Vivkzz avatar Apr 15 '25 04:04 Vivkzz

Can share some log?

asdacap avatar Apr 15 '25 04:04 asdacap