librdata icon indicating copy to clipboard operation
librdata copied to clipboard

Fix error in decompressing RDS files that decompress to more than UINT_MAX bytes

Open elaude opened this issue 1 year ago • 0 comments

When decompressing RDS files that decompress to more than UINT_MAX bytes (~4 GB), librdata incorrectly returns an error Could not open file.

This is caused by truncating assignment of an 8 byte ssize_t to a 4 byte uInt. I fixed this by doing saturating assignment instead. The surrounding decompression loop ensures that larger files can be decompressed (in chunks).

C.f https://github.com/madler/zlib/blob/develop/uncompr.c#L58 for a reference implementation that works this way.

I only fixed the bug that I could personally confirm fixed with my changes, the analogous functions using gzip or lzma decompression likely still have similar bugs.

Please consider turning on the -Wconversion flag for gcc, as this correctly warns about the assignment that caused this bug. When briefly turning on -Wconversion, I got a large number of warnings about other lines in the codebase.

elaude avatar Apr 22 '24 09:04 elaude