pyfilesystem2 icon indicating copy to clipboard operation
pyfilesystem2 copied to clipboard

Unable to use compress.write_zip() with s3fs source

Open james-emerton opened this issue 3 years ago • 0 comments

I believe this ticket should be for this package, but the issue is caused by implementation details of the fs-s3fs package. Attempting to create an archive of files from s3 using write_zip() fails with the following error:

Traceback (most recent call last):
  File "/Users/james/dev/spire/mount-s3-archive/test.py", line 7, in <module>
    write_zip(source, sys.argv[2])
  File "/Users/james/dev/spire/mount-s3-archive/.venv/lib/python3.10/site-packages/fs/compress.py", line 78, in write_zip
    mt = info.modified or datetime.utcnow()
  File "/Users/james/dev/spire/mount-s3-archive/.venv/lib/python3.10/site-packages/fs/info.py", line 322, in modified
    self._require_namespace("details")
  File "/Users/james/dev/spire/mount-s3-archive/.venv/lib/python3.10/site-packages/fs/info.py", line 128, in _require_namespace
    raise MissingInfoNamespace(namespace)
fs.errors.MissingInfoNamespace: namespace 'details' is required for this attribute

Code:

import sys

import fs
from fs.compress import write_zip

source = fs.open_fs(sys.argv[1])
write_zip(source, sys.argv[2])

Executing the above code as test.py s3://some-bucket/ test.zip will fail with the above error while a local source directory will compress nicely.

The issue seems to be that S3FS directory entries don't expose a details namespace. It's unclear if that's correct or not, but the code in write_zip() seems to assume that details is always available (but makes allowance for Info.modified to be a falsy value).

james-emerton avatar Oct 26 '22 23:10 james-emerton