Support file larger than 4GiB in 64-bit Notepad2
Loading large file is very smooth since 1815a8ed2b5ac6b8f6b336485ad60afa961a052c, large file TODOs taken from comment in EditLoadFile() 795fb952fcad443bb0b67e0990bc1810bbbb01aa:
- [x] [> 2 GiB] use
SC_DOCUMENTOPTION_TEXT_LARGEsomewhere or hard-coded inEditModel::EditModel(). - [ ] [> 4 GiB] use SetFilePointerEx() and ReadFile()/WriteFile() to read/write file.
- [x] [> 2 GiB] fix encoding conversion with MultiByteToWideChar() and WideCharToMultiByte().
- [x] [> 4 GiB] fix sprintf(), wsprintf() and wcstol() for Sci_Position and Sci_Line, currently UINT is used.
- [x] [> 2 GiB] ensure Sci_PositionCR is same as Sci_Position, see Sci_Position.h
Supporting for UTF-8 encoded file smaller than 4 GiB can simply implemented by replacing SC_DOCUMENTOPTION_DEFAULT with SC_DOCUMENTOPTION_TEXT_LARGE in EditModel::EditModel():
- pdoc = new Document(SC_DOCUMENTOPTION_DEFAULT);
+ pdoc = new Document(SC_DOCUMENTOPTION_TEXT_LARGE);
All other codes (except encoding conversion) already works for 4GiB file.
Since commit 538a8ca39459254e120da6303724bfb969a032fd, Notepad2 can loading file larger than 2 GiB (less than 4 GiB or 1/3 physical memory). Encoding conversion for file larger than 2 GiB is disabled, it will not be fixed: file larger than 2 GiB for encodings other than UTF-8 is rare.
From https://bugs.python.org/issue36311, it's suggested to use 512 MiB buffer to get better performance when using MultiByteToWideChar() for large input.
Larger than 4GiB file can be loaded easily with file mapping (mapping entire file is faster than reading entire file).
https://docs.microsoft.com/en-us/windows/win32/memory/creating-a-view-within-a-file https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createfilemappinga https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-mapviewoffile