filesystem_spec
filesystem_spec copied to clipboard
UnsupportedOperation error with ZipFileSystem using append mode
fsspec.implementations.zip.ZipFileSystem seems to have an issue with append mode 'a'. I assume, opening in mode 'a' allows for adding new Zip-entries to an existing Zip archive. At least this is what I need. While opening in append mode first seems to work - I can write to the archive - it will fail once the filesystem is closed.
While using mode 'w' works and creates an empty archive like so:
fs = fsspec.filesystem("zip", fo="test-w.zip", mode="w")
# write stuff here or do nothing...
fs = None # <-- force garbage collection of fs object
An error is raised when using mode 'a' once the filesystem is auto-closed by setting the reference to None or by calling fs.close() directly:
fs = fsspec.filesystem("zip", fo="test-a.zip", mode="a")
# write stuff here or do nothing...
fs = None # <-- error, also for fs.close()
Traceback:
Exception ignored in: <function ZipFileSystem.__del__ at 0x7fa07d5e4fe0>
Traceback (most recent call last):
File ".../lib/python3.11/site-packages/fsspec/implementations/zip.py", line 73, in __del__
self.close()
File ".../lib/python3.11/site-packages/fsspec/implementations/zip.py", line 78, in close
self.zip.close()
File ".../lib/python3.11/zipfile.py", line 1892, in close
self._write_end_record()
File ".../lib/python3.11/zipfile.py", line 1989, in _write_end_record
self.fp.truncate()
io.UnsupportedOperation: truncate
Reproducible on Linux and Windows.
Note, Python docs on zipfile.ZipFile state: If mode is 'a' and the file does not exist at all, it is created. If mode is 'r' or 'a', the file should be seekable.
I believe fsspec.implementations.local.LocalFileOpener may need to explicitly implement truncate().
I believe it should also be seekable if zipfile.ZipFile requires random-access (maybe in append mode only).
.seekable() is already explicitly implemented.
@martindurant , thanks! :)