mkfs: feature to output filesystem to stdout

[cedws]

I am using mkfs.ext4 to create a filesystem image, kind of like this:

$ mkfs.ext4 filesystem.img 1048576

This creates a file called filesystem.img that is 1048576 blocks in size.

Instead of creating a file, I would like the resulting filesystem to go to stdout so that I can pipe the result in directly to dd and avoid some heavy I/O. Some programs accept - as an argument indicating that the output should go to standard output but alas that has not been implemented here.

I thought I might be able to be clever by passing /dev/stdout as the filename but that didn't work either.

$ mkfs.ext4 /dev/stdout 1048576
mke2fs 1.46.2 (28-Feb-2021)
warning: Unable to get device geometry for /dev/stdout
Warning: could not erase sector 2: Illegal seek
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: cbb2efb4-ad58-424f-b678-5e3b2601d7ca
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376

Allocating group tables: done
Warning: could not read block 0: Illegal seek
Warning: could not erase sector 0: Illegal seek
mkfs.ext4: Illegal seek while zeroing block 262128 at end of filesystem
Writing inode tables: done
Creating journal (8192 blocks): mkfs.ext4: Illegal seek
	while trying to create journal

The error here might reveal a problem with writing output to stdout. The filesystem is not initialised as a continuous byte stream and requires seeking backwards and forwards - at least this is my assumption. I think it would be acceptable to buffer the whole filesystem in memory and dump the whole thing to stdout when complete.

I have also considered writing the filesystem to tmpfs or ramfs but the environment I'm working with is containerised and creating mounts is not possible.

[tytso]

The I/O required to do "mkfs.ext4 - 1048576 | dd of=/dev/sdXX bs=4k" is going to be far greater than simply doing "mkfs.ext4 /dev/sdXX 1048576". Specifically, writing to block device directly would require writing approximately 716k. dd'ing the image will requiring writing 1 gigabyte, or over 1400 times more block writes. So using dd is going to result way more heavier I/O --- it does not avoid heavy I/O --- rather the reverse.

[cedws]

Hello @tytso, this is because of sparseness right? It's worth mentioning that container runtimes don't handle sparse files. So writes to the underlying storage are heavy regardless of sparseness.

Related issue: https://github.com/moby/buildkit/issues/1749