nanos
nanos copied to clipboard
Disk I/O optimization
This change set aims at minimizing the number of I/O requests and scatter-gather buffers used to transfer a given amount of data to/from disk, which helps parallelizing I/O (especially when the I/O ring queue depth is limited) and minimizes the memory allocation needs for an I/O transfer. The first commit implements merging of contiguous pagecache pages that have their backing memory areas adjacent (which happens frequently if the cache for a node is filled sequentially starting from offset 0) into a single scatter-gather buffer before doing a filesystem storage operation (both when reading and writing a page cache node). The second commit implements extension of existing file extents to cover additional storage space beyond the initially allocated space, and removes the 1MB maximum extent size; this helps keeping the amount of file metadata down and minimizing fragmentation of file data into non-contiguous storage areas. Since allocations from the storage space are aligned to the allocation size, requesting large extents can create large unallocated ranges in the storage space; in order to be able to fill these ranges when requesting a new extent that does not fit into a single contiguous storage area, the create_extent() function upon allocation failure retries an allocation with a smaller size (down to a 1MB limit); the code that calls this function has been amended to properly handle the cases where the size of a created extent is smaller than requested. https://github.com/nanovms/nanos/issues/1165