Investigate adding MAP_FIXED support to Linux's nommu mm code

[davidchisnall]

Linux provides a variant of the the memory management code for non-MMU systems (mm/mnommu.c in the Linux tree). We could use that with LKL but it does not provide support for MAP_FIXED, which Java (among other things) depends on.

Once the only things calling the mmap system call are userspace things (everything other than userspace that needs to call the src/enclave version is doing so directly and not via musl), we can experiment with enabling this and see if everything that doesn't need MAP_FIXED works.

Supporting MAP_FIXED would probably not be too difficult for the limited set of cases that are needed. OpenJDK only ever needs MAP_FIXED | MAP_PRIVATE, so we don't have to handle complex cases where a process requests shared mappings at a fixed address, so a check that the fixed mapping is either in unused space or space owned by the calling process would be sufficient.

[davidchisnall]

Adding to the relayering project. This may or may not be required to avoid the lkl_run_in_kernel_stack code once we have replaced cryptsetup.

[prp]

David wrote:

I had a look in lkl/mm/nommu.c. If I found all of the relevant places in OpenJDK, it looks as if the logic in do_munmap contains the checks that we'd need to see if the current process owns a mapping to the region proposed with MAP_FIXED and, if so, we could allow it in the cases that OpenJDK requires. The code already supports MAP_SHARED by always returning the same location to all callers and MAP_PRIVATE by doing a new mapping and copying pages there. We can't support MAP_FIXED at an arbitrary location because, in uCLinux, the MMU code is layered above the kernel memory allocators, so it can request n pages, but it can't request a specific n pages. If the user has already made a mapping at a particular address, then we should be able to change the backing store though, as long as it either isn't MAP_SHARED or is the first MAP_SHARED mapping requested for a particular object (I think the latter is used by Java for making JIT'd code available to gdb, but not for anything else). The common use of MAP_FIXED is to punch a hole in an existing mapping and turn it back into CoW zeroes (in our case, just zero the range, though with EDMM we may want to do something more clever).

[prp]

Some findings:

• When removing the intercept for the mmap syscall in musl, it segfaults during the execution of the Stage 3 dynamic linker (here: https://github.com/lsds/sgx-lkl-musl/blob/92eaf58d34574d76745322af82c5f097c23e7b9f/ldso/dynlink.c#L1597).
• Calling the execve syscall fails due to the lack of binfmt_elf support in LKL, which requires an MMU (see https://patchwork.kernel.org/patch/7354161/).