kernel icon indicating copy to clipboard operation
kernel copied to clipboard

Investigate `tarfs` support (as part of image support)

Open fogti opened this issue 2 months ago • 6 comments

This came up during work on https://github.com/hermit-os/uhyve/issues/1015.

This would be a way to reduce possible future code duplication / duplicated work between loader and uhyve. The main alternative to this is:

  • uhyve implements "virtual file descriptors" to refer to read-only segments / blobs / slices of (decompressed tar) files. (this is now done, but mostly because it was intended to get done even independently of this)
  • loader implements a way to parse tar files and pass the file tree via FDT (device tree) to the kernel, the kernel would then handle this as a read-only file system. (this idea was ultimately rejected in favor of this)

cc @n0toose

fogti avatar Oct 22 '25 20:10 fogti

If we're lucky, one can (re)use most of the impl in hermit-image-reader (ThinTree::try_from_image)

fogti avatar Oct 23 '25 00:10 fogti

Why would we want this to be read-only and not implement it using the kernel's existing Ramfs/Romfs VFS nodes?

mkroening avatar Oct 23 '25 07:10 mkroening

romfs would probably be appropriate. The idea is to implement the initial parsing in the kernel, too, to avoid having to pass this information from the loader to the kernel, where we might run into troubles with lots of small string allocations.

fogti avatar Oct 23 '25 08:10 fogti

Are the small string allocations due to ThinTree::Directory(BTreeMap<String, ThinTree>)? Would it be possible to avoid intermediate allocations here and go from parsing to the device tree without the intermediate step? Or do tar files require some housekeeping before?

mkroening avatar Oct 23 '25 08:10 mkroening

The main problem with tar files is that file names require some processing as they might be present as two separate null-byte-free byte slices, which have to be joined by an /. Sure, one could implement a type that borrows instead and has an appropriate Display::fmt implementation, but passing this along is even more complicated.

fogti avatar Oct 23 '25 08:10 fogti

This probably depends on #1976, at least for now.

fogti avatar Oct 23 '25 13:10 fogti