Refactor metadata storage
Goals:
- Don't force user to specify metadata size.
- Allow metadata to grow dynamically instead of choosing a constant size up front.
Tasks:
- Split metadata into its own shared memory segment.
- Remove all metadata parameters from Hermes config file.
- Implement
Heap::realloc.
Dynamically resizing shared memory segments is difficult because we don't have any natural synchronization points in a running Hermes app outside of initialization and finalization. Synchronization is required because each process must remap it's shared memory segment after one process has grown the file via ftruncate. We can avoid this growing and remapping by giving the segment an initial max_metadata_size (configurable). Then we just map the whole segment. Based on some testing I did, only the blocks of the segment that we actually read/write are brought into physical memory (page cache actually), and once we're out of physical memory, those blocks are reclaimed (and dirty pages are written to disk). Therefore, mapping a 2GiB file doesn't put any pressure on physical memory, and it allows us to realloc within the segment, giving us dynamic maps, without app wide synchronization.