`prune`stats inconsistent / inaccurate

[jdannenberg]

What's the version of restic you used? Please include the output of restic version in your bug report.

restic 0.14.0 compiled with go1.19 on linux/amd64

What commands did you execute to get to where the bug occurred?

The starting point is a ~ 3,2 TiB repository pre 0.14.0. I then:

• updated restic and migrated the repository to v2
• ran: restic prune --compression max --pack-size 64 --repack-uncompressed --repack-small --no-cache
• This run got OOM killed at about 50-60%; I upgraded memory and continued
• ran: restic prune --compression max --pack-size 64 --repack-uncompressed --repack-small --no-cache --dry-run
• ran: restic prune --compression max --pack-size 64 --repack-uncompressed --repack-small --no-cache

I do not have any logs of the first restic prune run.

What did you expect?

I expected that prune dry-run and wet run behave the same (deterministic) regarding the stats they show. And I expect the numbers to be correct in any case.

What happened instead?

Log of 2nd prune:

loading indexes...                                                                                       
loading all snapshots...                                                                                 
finding data that is still in use for 842 snapshots                                                      
[38:23] 100.00%  842 / 842 snapshots                                                                     
searching used packs...                                                                                                                                                                                            
collecting packs for deletion and repacking                                                              
[7:12] 100.00%  736014 / 736014 packs processed                                                          
                                                                                                         
to repack:      **15759546 blobs / 3.151 TiB**                                                               
this removes:    1146545 blobs / 93.147 GiB                                                              
to delete:       **3723161 blobs / 1.768 TiB**                                                               
total prune:     4869706 blobs / 1.859 TiB                                                               
remaining:      14613001 blobs / 3.060 TiB                                                               
unused size after prune: **16777215.991 TiB** (100.00% of remaining size)

Log of 3rd prune:

loading indexes...                                                                                       
loading all snapshots...                                                                                 
finding data that is still in use for 842 snapshots                                                      
[35:23] 100.00%  842 / 842 snapshots                                                                     
searching used packs...                                                                                  
collecting packs for deletion and repacking                                                              
[7:44] 100.00%  736014 / 736014 packs processed                                                          
                                                                                                         
to repack:      **15828929 blobs / 3.154 TiB**                                                               
this removes:    1215928 blobs / 97.818 GiB                                                              
to delete:       **3653778 blobs / 1.765 TiB**                                                               
total prune:     4869706 blobs / 1.861 TiB                                                                                                                                                                         
remaining:      14613001 blobs / 3.058 TiB                                                               
unused size after prune: **16777215.989 TiB** (100.00% of remaining size)                                                                                                                                              
                                                                                                         
deleting unreferenced packs                                                                              
[23:54] 100.00%  88673 / 88673 files deleted                                                             
repacking packs                                                                                          
[11:53:12] 12.91%  86136 / 667243 packs repacked

There is a discrepancy between dry-run and wet-run in terms of blobs to repack and blobs to delete. Dry-run shows the respective blocks as to delete whereas wet-run shows them as to repack. Additonally, the unused size after prune is definitely inaccurate.

Are you aware of a way to reproduce the bug?

No.