Implement Retention Period Support

[coderofstuff]

Notes

• New configuration usage --retention-period-days=N - set this to some number to keep that many days of data. If this is smaller than the pruning period duration, a pruning period duration amount of data will be kept still.
• Implementation detail - this goes through the selected parent chain to find the first block that is past the N days threshold. There is no extra logic that ensures the anticone of the last SPC block found is also retained as such those will still be pruned.

[KaffinPX]

Instead of days why dont we keep it by pruning points, would be a more convenient way(for future archival p2p features)?

[coderofstuff]

Changes to apply:

• Instead of extra days, allow specifying exactly how many days to keep
• Track the history root independent of pruning point

[michaelsutton]

Changes to apply:

• Instead of extra days, allow specifying exactly how many days to keep
• Track the history root independent of pruning point

---

1. Also, search for the new history root from the bottom-up.
2. You might need to add another DB tracking field in addition to history root: source (or a better name). Flow:

• find new source (by walking up from current source and up the chain)
• set source = new source (right before prune starts)
• prune
• set history root = source

This way the code can safely use source in places where data validity is required (e.g., get_source). In recover_pruning_workflows_if_needed the condition for recovery should change to if history_root != source

[michaelsutton]

Instead of days why dont we keep it by pruning points, would be a more convenient way(for future archival p2p features)?

This might be a good idea, but we can apply this change later. I still think the cmd arg should be number of days to keep (perhaps in float?), and we can later translate that to the closest pruning point.

Btw, perhaps we should log on start-up with warning about approximate max disk usage (for transient data) given the requested days.

[michaelsutton]

Opened a PR over this branch with a few more essential leftover fixes I noticed (especially note the DB upgrade for archival cases): https://github.com/coderofstuff/rusty-kaspa/pull/19

[michaelsutton]

Ran some custom simpa tests with the following diff (and archival switched on and off). All looking good.

command line: cargo run --release --bin simpa -- -n=6000 -t=1 --test-pruning --loglevel=info

git diff
diff --git a/consensus/src/pipeline/pruning_processor/processor.rs b/consensus/src/pipeline/pruning_processor/processor.rs
index 923df58c5..c363c2eaa 100644
--- a/consensus/src/pipeline/pruning_processor/processor.rs
+++ b/consensus/src/pipeline/pruning_processor/processor.rs
@@ -205,6 +205,10 @@ impl PruningProcessor {
             // Inform the user
             info!("Periodic pruning point movement: advancing from {} to {}", current_pruning_info.pruning_point, new_pruning_point);
 
+            let virtual_daa_score = self.lkg_virtual_state.load().daa_score;
+            let retention_daa_score = self.headers_store.get_daa_score(adjusted_retention_period_root).unwrap();
+            warn!("VVVVVVVVVVVVVVVVVVV {}", virtual_daa_score - retention_daa_score);
+
             // Advance the pruning point utxoset to the state of the new pruning point using chain-block UTXO diffs
             if !self.advance_pruning_utxoset(current_pruning_info.pruning_point, new_pruning_point) {
                 info!("Interrupted while advancing the pruning point UTXO set: Process is exiting");
@@ -563,7 +567,11 @@ impl PruningProcessor {
                 let retention_period_ms = (retention_period_days * 86400.0 * 1000.0).ceil() as u64;
 
                 // The target timestamp we would like to find a point below
-                let retention_period_root_ts_target = unix_now().saturating_sub(retention_period_ms);
+                let retention_period_root_ts_target = self
+                    .headers_store
+                    .get_timestamp(self.lkg_virtual_state.load().ghostdag_data.selected_parent)
+                    .unwrap()
+                    .saturating_sub(retention_period_ms);
 
                 // Iterate from the new pruning point to the prev retention root and search for the first point with enough days above it.
                 // Note that prev retention root is always a past pruning point, so we can iterate via pruning samples until we reach it.
diff --git a/simpa/src/main.rs b/simpa/src/main.rs
index 3021ee568..f7f5a3749 100644
--- a/simpa/src/main.rs
+++ b/simpa/src/main.rs
@@ -199,6 +199,8 @@ fn main_impl(mut args: Args) {
         .apply_args(|config| apply_args_to_perf_params(&args, &mut config.perf))
         .adjust_perf_params_to_consensus_params()
         .apply_args(|config| config.ram_scale = args.ram_scale)
+        .apply_args(|config| config.retention_period_days = Some(0.01))
+        // .set_archival()
         .skip_proof_of_work()
         .enable_sanity_checks();
     if !args.test_pruning {