Add address option to MMapOptions

[tinco]

Hi, I'm implementing a memory allocator in Rust (porting the one from Go) and one requisite is being able to grow a specific mmap'ed vector without reallocating (for lockless purposes). Would be cool if I could use your crate for this, so I added this feature to it. I made an attempt at implementing the functionality for Windows as well, but I've no Windows box at my office so I haven't tested it. If you like the feature I'll make a better attempt at documenting and perhaps adding some tests.

[tinco]

Woops, forgot how making pull requests from the commandline worked :P

[danburkert]

Hi @tinco, AFAIK this isn't something that can be done reliably across platform, and as such it isn't something I want to support in this crate. In particular the behavior around what happens when the address is already in use by the process differs, e.g. on windows:

If the lpBaseAddress parameter specifies a base offset, the function succeeds if the specified memory region is not already in use by the calling process. The system does not ensure that the same memory region is available for the memory mapped file in other 32-bit processes.

Meanwhile mmap is happy to overwrite existing mappings, modulo the non-standard MAP_FIXED_NOREPLACE flag.

If you can find a way to make the behavior across platforms consistent then I'd be open to consideration. At a minimum, though, I think this new method would need to be marked unsafe (page replacement is deeply, deeply unsafe), tests for the cross platform behavior defined, and more documentation.

[tinco]

Hey Dan! Thanks for looking at the PR :)

I had not thought about overwriting existing mappings, it makes sense and I wouldn't mind at all making it be marked unsafe. I think that implies it should be done through a separate method, so not through MMapOptions then.

The data structure I'm trying to implement is actually ported from the Golang memory allocator, it has implementations of this exact thing for all manner of platforms including various unix flavours and windows. It uses VirtualAlloc instead of CreateFileMapping for this kind of map. I didn't want to disturb your code too much so I thought this would also work, but if this should be a separate method anyway that would probably be a better approach.

I'll take a look later and see if I can make it be truly cross platform.

[scriptandcompile]

Any progress on this? I would love to use this feature instead of shoe-horning my own (bad) version of this awesome library.

[tinco]

Hey! I moved my attention to another part of my code but I definitely want to continue on this, I'll have another look this weekend.

[npmccallum]

@danburkert Would you consider platform specific functions behind a feature flag? I too need the address parameter for the loader that I'm writing.

[josephlr]

@danburkert this is also something we would be interested in (allocating at a fixed address).

Initially, we could just have an address option that specifies at which virtual address a region will be mapped (but doesn't allow growing/overwriting allocations). This can be done completely cross-platform, and wouldn't need to be unsafe. This is the only functionality we would need.

A second (platform-specific option) would be to overwrite an existing memory mappings. This would require more work (i.e. changing to virtual alloc).