squashfs-tools icon indicating copy to clipboard operation
squashfs-tools copied to clipboard

mksquashfs pipe

Open i0x71 opened this issue 4 years ago • 3 comments

Hello, Is it possible to pipe put the output of mksquashfs to stdout ? I have half a terabyte of files that i would like to make into a squash image and pipe them into an s3 client, storing it in a local image file would take up unnecessary space.

Thanks

i0x71 avatar Apr 22 '20 23:04 i0x71

Try writing to /proc/self/fd/1

AnrDaemon avatar Jul 10 '21 06:07 AnrDaemon

No, the first several bytes (the superblock) depend on the results of building the whole archive.

Dr-Emann avatar Jul 10 '21 14:07 Dr-Emann

No, the output has to be seekable (which a pipe isn't).

This is first because the Squashfs superblock (which contains information about where things are placed in the filesystem and what is in it) is at the start of the filesystem and the contents are only known once the filesystem has been generated and written out. To update the superblock after the filesystem has been written out requires the output file to be seekable.

Second Mksquashfs does de-duplicating on the contents of the filesystem, and this again requires the output to be seekable, so Mksquashfs can read back previously written out files, and do a byte comparison.

There is way of doing it, but, it will require a reasonable amount of coding, and it got a lower priority to do than a lot of other requests. There is already a lot of new stuff in the next release, and it has taken a long time to test and document it.

So I intend to work on this feature after the next release (which be in the next couple of days).

plougher avatar Jul 11 '21 01:07 plougher