serenity icon indicating copy to clipboard operation
serenity copied to clipboard

Kernel+Userland: Introduce InitRAM filesystem support

Open supercomputer7 opened this issue 3 years ago • 4 comments

The main goal of this PR is to allow us to use SerenityOS on bare metal, without the need of writing a GRUB (or any other Multiboot-compatible bootloader) harddrive image to a physical PATA/SATA harddrive. This means we should be able to boot from a USB thumb drive, over PXE (or iPXE which is much more reliable), etc. This also means that we could run entirely from RAM, so the constant question of "why I can't use this with a USB thumb drive?" is eliminated. As a side effect, I expect this to raise the usage of Serenity on bare metal and as a consequence, to lead to more drivers being written.

For now, this PR is still incomplete. We can boot now directly to the initramfs image on i686 (x86-64 doesn't work yet due to low memory condition?). So the next thing I intend to work is to be able to swap the root mount with a real harddrive - this involves using a new syscall, called pivot_root (which is quite similar to what Linux has).

The current tasks to complete this are:

  • [ ] Fix the pivot_root syscall to be able to use a real harddrive and continue booting within it
  • [ ] Fix x86-64 setup to boot to both into the initramfs and a real harddrive with ext2 filesystem

This relies on #14932 and #14776. The inspiration to use the DROW ELF format is from #11368. Please note that I fixed most of the notes from that PR when I rebased it with my changes, but if you have any other suggestion, I'd be happy to hear about them :) Another PR being related to this is #14940, which intends to replace https://github.com/supercomputer7/serenity/commit/0ba59c3f26c1ee7fdc8ce564d8ecf63d31b943ff with a slightly better solution.

Preview of what has been achieved so far: Screenshot_2022-08-19_09-31-36 Screenshot_2022-08-19_10-58-41

supercomputer7 avatar Aug 20 '22 13:08 supercomputer7

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions!

stale[bot] avatar Sep 11 '22 13:09 stale[bot]

This pull request has been closed because it has not had recent activity. Feel free to re-open if you wish to still contribute these changes. Thank you for your contributions!

stale[bot] avatar Sep 18 '22 14:09 stale[bot]

Let's reopen this so we can actively advocate this to be merged as soon as possible :)

supercomputer7 avatar Oct 02 '22 02:10 supercomputer7

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions!

stale[bot] avatar Oct 23 '22 03:10 stale[bot]

This pull request has been closed because it has not had recent activity. Feel free to re-open if you wish to still contribute these changes. Thank you for your contributions!

stale[bot] avatar Nov 01 '22 02:11 stale[bot]

Now that the BuggieBox utility is being defined in #15373, we can try this again :)

supercomputer7 avatar Nov 19 '22 03:11 supercomputer7

So now that it works quite beautifully, I will post some pictures so you can view the environment: Screenshot_2022-11-19_19-53-55 Screenshot_2022-11-19_19-54-27 Screenshot_2022-11-19_19-55-51

supercomputer7 avatar Nov 19 '22 17:11 supercomputer7

Leaving a side note here - if we failed to boot and we were dropped into emergency shell, and then continued booting with init /dev/hdX (or something like this), the graphical environment boots fine up until the LoginServer, but you can't login. If we weren't dropped into the emergency shell (because the root= kernel command-line option was set correctly) or we use textual mode - everything behaves normally.

supercomputer7 avatar Nov 19 '22 17:11 supercomputer7

Thinking about it seriously, the whole pivot root syscall should really look different. The main problem is our ā€œmount tableā€, in particular the fact that it is a Vector and not an intrusive list - cant see one reasonable advantage for this, so let’s shave the yak here and make it right - it will also ensure that we don’t have to awkwardly enumerate the list like the way we do in many places in the VFS code.

supercomputer7 avatar Nov 21 '22 17:11 supercomputer7

This has conflicts.

AtkinsSJ avatar Dec 06 '22 15:12 AtkinsSJ

There are a few roadblockers to this

  • Getting rid of i686 support: This will greatly help to not run out of memory in case of big initramfs images, as I'd like to restore TmpFS "free" allocation of virtual memory
  • If we rely on CPIO for the initramfs image, I'd like to get a utility for handling that format first.
  • If we rely on other archive format for the initramfs image, I'd like to ensure that everything is intact with our implementation. Options are SquashFS, CramFS, EROFS, tar, zip, BootFS, FAT16/FAT32 or ROMFS. Preferably, the solution should easy to implement in our project and that has all the required utilities for Linux.
  • We need to determine whether compression of the initramfs image is important or not. For the first iteration we probably could be OK without it, but I imagine that people would like to run a full-blown OS from RAM (and with today personal computers which have even 32 GB of RAM and sometimes 64 GB, this is completely a sane goal) and to make it possible (downloading an image of 500 MB instead of 4 GiB), compression is completely unavoidable and is necessary to ensure that. It will be important to determine if either support compressed and uncompressed images or only one of the option.
  • We still need to work out the pivot_root syscall. It works, but I am not very happy with the result.
  • BuggieBox needs more utilities within it, so in case of failure, you could do something meaningful with it
  • In general, I'd like to ensure it all feels natural to the project. The goal is very positive, but if it will slow things down during boot, or the ergonomics will not fit, then we will need to fix this.

supercomputer7 avatar Dec 19 '22 10:12 supercomputer7

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions!

stale[bot] avatar Jan 27 '23 21:01 stale[bot]

We can close this, as #16855 will resolve this as well.

supercomputer7 avatar Feb 11 '23 08:02 supercomputer7