demikernel icon indicating copy to clipboard operation
demikernel copied to clipboard

[catmem] huge page support

Open kyleholohan opened this issue 2 years ago • 0 comments

Context

Huge pages offer benefits for some high resource use applications. Applications which transmit large amounts of data via shared memory may benefit from using huge pages for IPC. Per the Linux kernel documentation on huge pages, huge pages exist as files mounted under hugetlbfs filesystem. shm_open creates files on a tempfs usually under /dev/shm.

Windows also supports huge pages (called large pages); use of MapViewOfFile3 allows use of the MEM_LARGE_PAGES flag, as well as optional NUMA zone placement through ExtendedParameters.

Proposed Solution

Move the PAL away from usage of shm_create so that files can be put either on tempfs or hugetlbfs. Create a new abstraction for resolving shared memory file names on the appropriate device. Add one implementation for tempfs and one for hugetlbfs.

For Windows in conjunction with #965, add the ability to use MEM_LARGE_PAGES when mapping shared memory.

Alternative Solutions

Adding this feature is a pretty binary choice. Huge pages will likely only offer benefits for a small subset of the application space. This should be driven accordingly.

For implementation, the current PAL implementation could instead be preserved and a hugepage PAL could be added. This would result in some redundant code, but more stability for existing cases.

Edit

I missed a subtlety here: shm_open is the POSIX-compliant interface to shared memory, and huge pages don't seem to exist in the POSIX standard. In this case, preserving the existing PAL may serve value for POSIX-compliant non-Linux targets.

kyleholohan avatar Oct 10 '23 20:10 kyleholohan