[2.0.0b9] archive stats always 0

[holomekc]

Hi,

first of all thx for all the effort you guys put into this project.

I am playing around with borg 2.0.0b9 at the moment. I noticed that the stats look strange. Maybe it is not implemented yet?

The deduplication_size is always 0. The same applies to other stats:

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

2.0.0b9

Operating system (distribution) and version.

macOS Sonoma 14.5

Hardware / network configuration, and filesystems used.

Just locally playing around, but M2 Pro

How much data is handled by borg?

Not much. You can see in the screenshots.

Full borg commandline that lead to the problem (leave away excludes and passwords)

Repo is set via env variable

borg create <archive-name> .../borg-source --info --progress --show-rc -s borg info --last 20 --json

[ThomasWaldmann]

Some breakage with the stats is expected due to the new AdHocWithFilesCache cache implementation.

The old LocalCache implementation did precise refcounting and also had the chunk size in the chunks cache. But it needed extra effort to keep that cache coherent with the repo, e.g. in multi-client usage.

The new implementation does not do precise refcount usually (just knows whether we have a specific chunk or not) and it also does not know the size of chunks in the chunks cache.

That is implemented in a way that is somehow compatible with the other implementation and uses kind of "infinite" reference count so it newer goes to zero and the size is just 0, which normally can't be. That way, both implementation can work with the chunks cache contents without anything important going wrong (like e.g. it deleting something that is still needed).

But, one side effect is that the stats are incorrect when using the new implementation, so we need to just ignore them for now.

In general, I am not very happy with the stats in borg:

• they are all over the place in the borg code and borg does a big effort (in code complexity and also runtime cpu and IO) and tries to be very precise updating these stats
• same applies for the repo quota
• for many cases, this could be done with much less effort and it would be still good enough (like a du value for the whole repo dir, plus maybe the total size of freshly uploaded data)

[ThomasWaldmann]

So, yes, that is kind of a bug, but the solution might be not fixing it, but doing it differently.

[ThomasWaldmann]

Guess beta 10 will be also quite a bit "off" concerning stats at misc. places, because I ripped out a lot of code that was mostly "in the way".

Good news: b10 has borg compact -v and that displays some interesting stats about overall compression, overall deduplication, space usage, how much space it freed, object count, etc.

[ThomasWaldmann]

7a93890602cb3b1434f46076d83299a57de8a643 removed the "deduplicated size" computation completely, also the "Deduplicated size:" from borg info output was removed.

It was not that useful anyway and hard to understand (and often misunderstood).