Dmitry Litvintsev

Ideally. If the file associated with failed transfer were removed, the second issue would not have happened. Second issue is NFS specific and can be classified "why don't we fail...

Statistically the error is rather noticeable. Here us daily counts: ``` day | count ------------------------+------- 2022-01-13 00:00:00-06 | 1 2022-01-14 00:00:00-06 | 378 2022-01-15 00:00:00-06 | 1538 2022-01-16 00:00:00-06 |...

I went ahead and looked at all these failures (using billing DB). Many of the files have Cached (non broken) replicas. I can cp these files with no issues. Now...

I am just saying that some of the files that failed with this error end up with Broken replicas and some of these files end up with OK replicas. And...

Yes this is correct. Example: ``` [fndca3b] (PnfsManager@namespaceDomain) enstore > \sl 0000CBC5A7DCE51F43F5860E5B982A4D0B92 rep ls 0000CBC5A7DCE51F43F5860E5B982A4D0B92 p-sbn-stkendca2028-5: 0000CBC5A7DCE51F43F5860E5B982A4D0B92 29619695 si={sbn.persdata} ``` Billing: ``` billing=# select d.datestamp, b.protocol, b.datestamp, d.errorcode, d.errormessage, b.errorcode,...

Just to re-iterate What happened - clients switched from GFTP to HTTPs as default. And this is the most noticeable aftermath. When using GFTP there has always been transfer failures...

I am pretty sure that this is not chunked transfers They just upload files from worker nodes to dCache after job has finished. I do know about `Content-Lengh` header. I...

Statistically speaking this is breakdown of all errors for Http protocol for today so far: ``` count | errormessage -------+----------------------------------------------------------------------------------------- 31022 | No connection from client after 300 seconds. Giving...

[www_cp.sh.txt](https://github.com/dCache/dcache/files/8028239/www_cp.sh.txt) OK, got hold of client side log, looks like so: ``` ifdh cp -D gm2DQC_ana_29037383_44371.00255.root /pnfs/GM2/scratch/users/gm2pro/prod/DQCAna/5304A/Run4QuadRF-v9_74/2022-02-03-02-05-30/data Thu Feb 3 08:18:38 2022 program: www_cp.sh /srv/gm2DQC_ana_29037383_44371.00255.root https://fndca1.fnal.gov:2880/GM2/scratch/users/gm2pro/prod/DQCAna/5304A/Run4QuadRF-v9_74/2022-02-03-02-05-30/data/gm2DQC_ana_29037383_44371.00255.rootexited status 55 delaying 0...

Yes it is logged. The script is called with these arguments: ``` program: www_cp.sh /srv/gm2DQC_ana_29037383_44371.00255.root https://fndca1.fnal.gov:2880/GM2/scratch/users/gm2pro/prod/DQCAna/5304A/Run4QuadRF-v9_74/2022-02-03-02-05-30/data/gm2DQC_ana_29037383_44371.00255.root ``` local source, remote destination. So only ` -f -L --silent --capath .. --key...