dcache icon indicating copy to clipboard operation
dcache copied to clipboard

QoS Turns Files PRECIOUS again after Flush to Tape

Open christianvoss opened this issue 1 month ago • 0 comments

Hi all,

we've observed a strange behaviour recently. Files with QoS disk+tape have been made PRECIOUS again even after a flush and remain so even after a manual verify in the admin interface or even after a migration move.

We see the flush of a particular file:

{"instance": "cms", "host_type": "pool", "host": "dcache-cms239", "Domain": "dcache-cms239-04Domain", "date": "2025-12-08T02:21:33.273Z", "message": "[] Changing state of 0000F2EC85D5CEA041BCA85143245D2DE524 from PRECIOUS to CACHED: file sucessfully flushed", "cellName": "dcache-cms239-04", "pid": 708037}

while shortly afterwards, QoS sets the file again to PRECIOUS

{"instance": "cms", "host_type": "pool", "host": "dcache-cms239", "Domain": "dcache-cms239-04Domain", "date": "2025-12-08T02:21:45.143Z", "message": "[QOS-56d41442-ba26-4196-a89d-0f686bd80846 qos-adjuster PoolMigrationCopyReplica 0000F2EC85D5CEA041BCA85143245D2DE524] Changing state of 0000F2EC85D5CEA041BCA85143245D2DE524 from CACHED to PRECIOUS: migration request from qos-adjuster@dcache-cms-qos01_QoS-adjusterDomain", "cellName": "dcache-cms239-04", "pid": 708037}

For another file we tried to trigger a manual check:

[dcache-cms-core01] (local) vossc > \s qos-verifier verify ls 00007729AA5E141442FEB5625818A7FFCC09
[dcache-cms-core01] (local) vossc > \sl 00007729AA5E141442FEB5625818A7FFCC09 rep ls 00007729AA5E141442FEB5625818A7FFCC09
dcache-cms237-02:
    00007729AA5E141442FEB5625818A7FFCC09 <-P------X--L(0)[1]> 250950793 si={cms:user}

[dcache-cms-core01] (local) vossc > \sn storageinfoof 00007729AA5E141442FEB5625818A7FFCC09
size=250950793;new=false;stored=true;sClass=cms:user;cClass=-;hsm=cta;accessLatency=ONLINE;retentionPolicy=CUSTODIAL;links=0000926291B0C0FC44A797A93DD3BE6BF693 18ParkingBPH5_RunD_2021UL_5832.root;cta://cta/00007729AA5E141442FEB5625818A7FFCC09?archiveid=74645683;store=cms;group=user;bfid=<Unknown>;

As can be seen, the QoS would be disk+tape but the file itself has a valid tape location. Still, the verification finished and the file remains precious. A migration move does not change it either.

[dcache-cms-core01] (dcache-cms237-02@dcache-cms237-02Domain) vossc > \sl 00007729AA5E141442FEB5625818A7FFCC09 rep ls 00007729AA5E141442FEB5625818A7FFCC09
dcache-cms278-02:
    00007729AA5E141442FEB5625818A7FFCC09 <-P------X--L(0)[1]> 250950793 si={cms:user}

We're running

[root@dcache-cms-qos01 ~]# dcache version
10.2.18

for the QoS service and pools.

Cheers, Christian

christianvoss avatar Dec 10 '25 11:12 christianvoss