issues after upgrade - nfexpire?

[thezoggy]

upgraded nfdump from:

nfdump -V
nfdump: Version: 1.7.4-9198d94 Options: ZSTD BZIP2 Date: 2024-03-17 17:52:55 +0100

to

nfdump -V
nfdump: Version: 1.7.4-564f3a9 Options: ZSTD BZIP2 Date: 2024-05-11 10:55:50 +0200

changes: https://github.com/phaag/nfdump/compare/9198d94..564f3a9

I upgraded nfdump at 2024-05-16 ~1600 UTC, and while things were fine initially but per graphs there you can see it struggled up until a day later box ran out of hdd space..

Was there any recent changes to to the nfcapd format or something that would make nfexpire stop working?

I rebuilt profile data, picked a device and lowered its retention from 14d to 12d and ran nfexpire on it and saw it did drop some data. Reset that device back to max life.. got nfsen back up and changed max size from 38T to 37T for the profile. Then looks like nfexpire did run and data dropped down to 34T and all seems like its working.

> sudo /data/nfsen/bin/nfsen -l live
name	live
group	(nogroup)
tcreate	Wed Feb  8 17:50:00 2023
tstart	Sun May  5 22:30:00 2024
tend	Fri May 17 22:40:00 2024
updated	Fri May 17 22:40:00 2024
expire	14 days 0 hours
size	33.6 TB
maxsize	37.0 TB
type	live
locked	0
status	OK
version	130

So I can only gather that after upgrading nfdump, some reason nfexpire stopped working and over the course of the following 26.5 hours it gobbled up all free space until everything really broke when the box had 0 free space.

I'd write it off as a gremlin but now here an hour after I fixed it I see that its already at 91% full and showing the banner warning on nfsen. Which normally do not see due to the free space overhead.

Which makes me curious if its just bad timing of our overhead just been dwindling and we did not know and the upgrade just made things worse because of a data format change that makes it take up slightly more data to the point it trips the warning.. or if stuff truely just stop expiring like normal until we have to do manual commands or something.