root-project
Overcome 1GB size limit for IO buffers
Explain what you would like to see improved
ROOT currently has a size limitation (~1GB) for the number of bytes that can go into a single entry of a TBranch. This limitation is due to using code based on 32bit integers in the IO mechanism.
For Run3, the ALICE experiment is turning from an event (singe collision) - based data processing to a processing based on event-collections (timeframes). A timeframe becomes the basic data index. In consequence this puts us into a situation in which we often need to store/serialize very large amounts of data per TBranch entries.
So far, we have worked around this by either giving up on ROOT serialization features, or writing specialized IO containers. But it is foreseeable that the problem aggravates and a fix in the ROOT ecosystem would be a much better solution.
Optional: share how it could be improved
Please improve on the ~1GB size limitation, which is does not correspond to modern memory capabilities and 64bit processing. As a short term solution, gaining a factor of 2 would already help us. This might be possible by reviewing the use of int32 vs uint32 etc.
To add my opinion: I think this is quite a central ROOT feature, and supporting larger buffers will become imperative at some point anyway, so I don't see a strong reason to delay such changes. If backward compatibility is a concern, I think it would even be possible (or acceptable for ALICE) to create the file with a special flag or so, such that it cannot be opened with old ROOT versions. At least the current behavior seems very outdated to me. The first time I ran into this 1GB limit, it was actually the last thing I thought of to suspect the buffer size, in particular since the limit is 1 GB not 4 GB.
