Is it possible to add a last chance option for file transfers via FTS ?

[marianne013]

I'm currently seeing the following problem: An FTS transfer fails:

(base) lx05:move_moedal > dirac-rms-request 800512 | grep /vo.moedal.org/user/a/aditya.upreti/BeamPipe/gen/SchGen_GRID_80GeV_50gD_5000ev_10611_May20.mac
    [469] ID=7552636 LFN='/vo.moedal.org/user/a/aditya.upreti/BeamPipe/gen/SchGen_GRID_80GeV_50gD_5000ev_10611_May20.mac' Status='Failed' Checksum='2e4b1535'

looking at the log file on the FTS server:

https://fts01.grid.hep.ph.ic.ac.uk:8449/var/log/fts3/transfers/2024-06-17/se03.esc.qmul.ac.uk__gfe02.grid.hep.ph.ic.ac.uk/2024-06-17-1324__se03.esc.qmul.ac.uk__gfe02.grid.hep.ph.ic.ac.uk__52380956__33752f0c-2ca9-11ef-8adf-525400e091e6

It ends in:

INFO    Mon, 17 Jun 2024 14:24:02 +0100; DESTINATION Source and destination file size matching
INFO    Mon, 17 Jun 2024 14:24:02 +0100; Transfer finished successfully
INFO    Mon, 17 Jun 2024 14:24:07 +0100; Finishing copy process...

But the "global" message is:

Error reason: No FTS server has updated the transfer status the last 900 seconds. Probably stalled

Now the file is at Imperial (correct size and all):

[lt2-moedal001@gfe02 moedal]$ pwd
/pnfs/hep.ph.ic.ac.uk/data/moedal
[lt2-moedal001@gfe02 moedal]$ ls -l vo.moedal.org/user/a/aditya.upreti/BeamPipe/gen/SchGen_GRID_80GeV_50gD_5000ev_10611_May20.mac
-rw-r-----. 1 lt2-moedal001 lt2-moedal 19758 Jun 17 15:50 vo.moedal.org/user/a/aditya.upreti/BeamPipe/gen/SchGen_GRID_80GeV_50gD_5000ev_10611_May20.mac

But not registered in the file catalogue:

(base) lx05:move_moedal > dirac-dms-lfn-replicas /vo.moedal.org/user/a/aditya.upreti/BeamPipe/gen/SchGen_GRID_80GeV_50gD_5000ev_10611_May20.mac
Replica Lookup Time: 3.65 seconds 
LFN                                                                                            StorageElement     URL
=====================================================================================================================
/vo.moedal.org/user/a/aditya.upreti/BeamPipe/gen/SchGen_GRID_80GeV_50gD_5000ev_10611_May20.mac UKI-LT2-QMUL2-disk srm://se03.esc.qmul.ac.uk:8444/srm/managerv2?SFN=/vo.moedal.org/vo.moedal.org/user/a/aditya.upreti/BeamPipe/gen/SchGen_GRID_80GeV_50gD_5000ev_10611_May20.mac

Is there any way for the transfer software to do one last ditch check to see if the file has actually transferred and the register it ?

Probably a question for @chaen and and FYI for @sfayer

[chaen]

As discussed live:

• the FTSAgent should retry, so why it did not is unknown to me
• the dirac-rms-request --Reset is what you need