starterkit-lessons icon indicating copy to clipboard operation
starterkit-lessons copied to clipboard

DiracFiles

Open mesmith75 opened this issue 6 years ago • 2 comments

For some reason new members are using LocalFile for their tuple outputs when running on the grid. The advice for many years has been to store job output on the grid.

This needs to be made clear in the starterkit.

mesmith75 avatar Mar 20 '19 11:03 mesmith75

We do say in the More Ganga lesson:

To avoid filling up the filespace, it is wise to put the large files produced by your job somewhere with lots of storage - the grid.

But we do say in the "Storing large files on EOS" lesson

To retrieve a job outputfile, one can use two types of files:

  • LocalFile: the standard one with the output file directly downloaded to the gangadir.
  • DiracFile: the output file is stored directly on the grid and be accessed through the XRootD protocol.

We could add a caveat here (or in both places) to highlight that LocalFile is intended only for small (< 10 MB?) files.

alexpearce avatar Mar 20 '19 14:03 alexpearce

This lesson uses a LocalFile with the grid. I guess people might copy from there without realising.

chrisburr avatar Mar 20 '19 14:03 chrisburr