swift-system
swift-system copied to clipboard
Implement `FileDescriptor.memoryMap`, `FileDescriptor.memoryUnmap`, and `FileDescriptor.memorySync`
Add Swift implementations for mmap, munmap, and msync:
let swiftString = "swift"
let map = try fd.memoryMap(length: swiftString.count, pageOffset: 0, kind: .shared, protection: [.write])
let ptr = map.assumingMemoryBound(to: String.self)
// Update the mapping
ptr.pointee = swiftString
// Flush changes to the file system
try fd.memorySync(memoryMap: map, length: swiftString.count, kind: .synchronous)
// Delete the mapping
try fd.memoryUnmap(memoryMap: map, length: swiftString.count)
// Read in the file and validate the string was written correctly
let readBytes = try Array<CChar>(unsafeUninitializedCapacity: swiftString.count + 1) { (buf, count) in
count = try fd.read(into: UnsafeMutableRawBufferPointer(buf))
}
XCTAssertEqual(String(validatingPlatformString: readBytes), swiftString)
This PR adds all three methods because the ability to sync and delete mappings are sensible requirements of being able to create them.
Add FileDescriptor.MemoryProtection to describe the desired memory protection of a mapping. The memoryMap method accepts these values as an array to be bitwise OR'd by the implementation.
Add shared and private for FileDescriptor.MemoryMapKind. There are more C flags available, but this seemed like the simplest starting point.
Add synchronous and asynchronous FileDescriptor.MemorySyncKind. The memorySync method accepts either one of these values and an invalidateOtherMappings boolean that is bitwise AND'd in the implementation to represent the appropriate C flag.
Add internal constants for flags/values related to mmap, munmap, and msync.
Add additional methods for more generic mocking and error handling. This seemed like a good addition for system calls that do not return CInt.
This PR does not include a Windows implementation.
Open question: Should memory mappings be a unique type?
Arguments in favor:
- Memory mappings can outlive FileDescriptors.
- Unmapping an
UnsafeMutableRawPointerleaves you with an unusable instance with no way to ascertain that it has been unmapped.
Arguments against:
- For the most elegant API for this, it would add a series of structs/overhead: probably a
MemoryMapRawandMemoryMap<T>that have a very similar API toUnsafeMutableRawPointerandUnsafeMutablePointer<T>. The added methods would besync,unmap, andisUnmapped. This is more precise than what currently exists in this PR at the cost of overhead.
There is this whole family of address space-related functionality that we can't currently group under a specific type. This includes the anonymous flavor of mmap, as well as munmap, msync, madvise etc. (It would also be great if we could expose a way to enumerate existing memory mappings, but I have no idea if there are public APIs for that, or even if that would be a good fit for System.)
@milseman are you okay with just pulling these in as top-level functions?
We could choose to instead define a namespacing enum, and add all of these as members:
enum Memory {
static var pageSize: Int { get }
// properties for getting the cache line size, cache sizes etc.
func map(at:, length:, protection:, flags:, from:, offset:) throws -> UMRP
func unmap(at:, length:) throws
func protect(at:, length:, protection:) throws
func sync(at:, length: flags:) throws
// BSD/Darwinisms
func advise(at:, length:, advice:) throws
func inherit(at:, length:, inherit:) throws
func isInCore(at:, length:) throws -> ??? // mincore (BSD)
// Realtime stuff
func lock(at:, length:) throws
func unlock(at:, length:) throws
func lockAll(flags:) throws
func unlockAll() throws
}
This could be an attractive option, if for no other reason but to move these really low-level functions out of highly visible namespaces, such as FileDescriptor. (I believe (hope?) the number of Swift programs that have to use file descriptors is likely to heavily outweigh those who need to deal with memory mappings. I could be wrong, though.)
For sysconf, we have https://github.com/apple/swift-system/pull/33, which we can pull stuff from any time. Those sketches also have the general need for a namespace for global functions. Should we have many or just one? E.g. enum FileSystem {}, enum SystemConfig {}, ...
Oh, good point! I forgot we wanted to introduce SystemConfig for that. The page size etc. information feels like a good fit for it.
I think if we add one of these, we may as well add as many as we need.
(I remember being worried about adding file operations like copy/rename/remove into a FileSystem namespace rather than having them directly on FilePath, because that is such a convenient place for them. But then again, perhaps those convenience functions would be better defined in a separate module, with System restricting itself to basic syscalls like creat/unlink/rmdir/rename etc, which could just as well go under a namespace.)