pyfilesystem icon indicating copy to clipboard operation
pyfilesystem copied to clipboard

xrootd filesystem implementation

Open lnielsen opened this issue 10 years ago • 2 comments

We have developed an xrootd filesystem implementation for PyFilesystem, and is wondering if it's possible to link to our implementation from the PyFilesystem documentation. Most natural would probably be to add it in the section "Third-Party filesystems", however all the current entries references implementations which are distributed with PyFilesystem. It doesn't really makes sense to include the source into PyFilesystem since it has external dependencies which are likely only needed by few.

XRootD

XRootD protocol aims at giving high performance, scalable fault tolerant access to data repositories of many kinds. The protocol is heavily used in High Energy Physics and is the primary protocol for accessing physics data from CERN's Large Hadron Collider (~100 PB and growing).

XRootD allows for e.g. parallelised copying, random access to files and parallelised reading of chunks among other things.

xrootdfs

The PyFilesystem implementation is a small wrapper around the xrootd C client library itself to ensure compatibility as well as high performance.

Links:

  • https://pypi.python.org/pypi/xrootdfs
  • http://pythonhosted.org/xrootdfs/
  • https://github.com/inveniosoftware/xrootdfs/

lnielsen avatar Sep 29 '15 10:09 lnielsen

What if we added a new "External filesystems" section to the documentation, which covered filesystem implementations that work with PyFilesystem, but that aren't distributed with PyFilesystem itself?

lurch avatar Apr 27 '16 15:04 lurch

Solution seems fine to me. I can make a PR with it.

lnielsen avatar Apr 27 '16 18:04 lnielsen