squashfs-tools icon indicating copy to clipboard operation
squashfs-tools copied to clipboard

Enhancement: mksquashfs from pseudo-file pipe

Open svet-b opened this issue 2 years ago • 1 comments

This is the equivalent to https://github.com/plougher/squashfs-tools/issues/25, but for input from a pseudo-file rather than tar.

At the moment running mksquashfs - <output> -pf <pseudo-file> appears to only work for a seekable files. I.e. the error

FATAL ERROR: Lseek on pseudo file /dev/fd/XX failed because Illegal seek

occurs if trying to use a pipe for the pseudo-file.

Hopefully since piped tar is now acceptable, it wouldn't be a stretch to also accept piped pseudo-files?

The use case here is similar to what's described in Section 17 of the README. We want to take an image, make some modifications, and re-save it as a new image - ideally on the fly. Since we can pipe output to a pseudo-file in the first step and edit the pseudo-file stream in the second step, it would be great to be able to simply pipe back to a SquashFS image as the last step. At the moment this requires saving a temporary pseudo-file. It would be great to avoid that last requirement.

svet-b avatar Apr 21 '22 16:04 svet-b

It could be made non-seeking, but, in doing so it will remove many of the editing operations possible at the moment.

This is an example Pseudo file generated by Unsquashfs

	/ D 1625536033 1777 0 0
	test D 1625536969 755 1000 100
	test/hello_world R 1625535696 644 1000 100 12 0
	test/regfile R 1625536928 644 1000 100 37 12
	test/symlink S 1625535712 777 1000 100 hello_world
	#
	# START OF DATA - DO NOT MODIFY
	#
	Hello World
	This is the data contents of regfile

There are two regular files:

  1. First file "hello_world" with its data at offset 0
  2. Second file "regfile" with its data at offset 12

With the unmodified Pseudo file it is possible to process it in a non-seeking manner. The data of the first file met walking the constructed file hierarchy (hello_world) is the first data in the data-section, the data of the second file met is the second data in the data-section. The data is placed in the correct order exactly where and when it is needed.

hello_world is alphabetically before regfile in the test directory, which is why it is discovered first.

Some one should be able to edit the "hello world" name to be say "salve_mundi". It is now alphabetically after regfile, and now the data-section is not ordered in the way the files are created. To handle this would require a seek to location 12 for regfile, and a seek back to location 0 for "salve_mundi".

To prevent the data-section from becoming unordered, would mean files and directories can't be renamed so that it is always in the "right place".

If your editing doesn't involve renaming, then a "restricted" piping mode could be supported.

plougher avatar Aug 02 '22 03:08 plougher

Fixed in https://github.com/plougher/squashfs-tools/commit/6b556fd122f3934e34f4b2e993ce32fd8b773838 FIxed in https://github.com/plougher/squashfs-tools/commit/bf464a2d86caa21cd530dc8398513d74ca6e5ac9 Fixed in https://github.com/plougher/squashfs-tools/commit/592fe7b4095c96e72348d7d388d251fd94060ff6

plougher avatar Oct 15 '22 10:10 plougher