STL
STL copied to clipboard
microsoft
<streambuf>: basic_streambuf doesn't support 64-bit properly
std::basic_streambuf doesn't support 64-bit properly, because it holds the counters in 32-bit ints:
https://github.com/microsoft/STL/blob/b61483432979aafb6b89b245fb8e82eee7aee32f/stl/inc/streambuf#L392-L395
Then e.g. in std::basic_streambuf::setg(...) you have:
https://github.com/microsoft/STL/blob/b61483432979aafb6b89b245fb8e82eee7aee32f/stl/inc/streambuf#L210
and forcing a cast to int prevents from using with a memory range bigger than ~2GB (even on Win64 builds).
Also tracked by DevCom-102668, DevCom-290124, DevCom-320893, DevCom-859581, and Microsoft-internal VSO-485517 / AB#485517.
vNext note: Resolving this issue will require breaking binary compatibility. We won't be able to accept pull requests for this issue until the vNext branch is available. See #169 for more information.
@JeanPhilippeKernel vNext branch is not available yet and as @StephanTLavavej said in #290 it probably won't be before the second half of calendar year 2020.
Yeah - we're eager to resume work on vNext but first we have to finish migrating our build/test infrastructure, and then we need to manually port the changes we accumulated for vNext in a Team Foundation Version Control branch years ago (the codebase has evolved considerably since then, so the port will be time-consuming). Thanks for your patience, we hope vNext will be worth the wait :-)
I have just found out our library is affected by exactly that same issue. Do you recommand a workaround to bypass this limitation while waiting for the fix to be implemented?
I seem to have gone around it by changing our custom streambuffer class to always use a read range of at most std::numeric_limits<int>::max() bytes, and override underflow() to move this range further whenever more bytes need to be read. Feels a bit hackish though.
I'm not familiar enough with iostreams to say whether such a workaround is reliable (despite working on this codebase for 13 years). @BillyONeal may be able to comment, as he's delved into iostreams' arcane mysteries.
I seem to have gone around it by changing our custom streambuffer class to always use a read range of at most
std::numeric_limits<int>::max()bytes, and overrideunderflow()to move this range further whenever more bytes need to be read. Feels a bit hackish though.
To the best of my understanding that should work (it's similar to how the stream buffers exposed don't contain all a file's contents) but we can't do it to the built in ones without breaking ABI.
Thank you, we'll go for that workaround then. It passed all our tests, but of course they can't cover every possible case.
Found this out the hard way…
I ended up overwriting xsputn and using my own pointer and length variables when inheriting from streambuf. It's enough for my use-case (ostream->pre-allocated memory buffer)
Please make sure this gets fixed in the next ABI version as 2GB isn‘t much these days.
Any update on this ? see also my comment at : https://github.com/microsoft/STL/issues/578#issuecomment-1849502832
