flatbuffers
flatbuffers copied to clipboard
[C++] Fix vtable deduplication for 64-bit buffers >2GB
Fixes #8590
Summary
This PR corrects vtable deduplication for FlatBuffers using 64-bit offsets. Previously, when serializing buffers larger than 2GB, vtable deduplication failed, resulting in much slower serialization and larger buffer sizes.
Root Cause
The issue was caused by 64-bit offset calculations not accounting for length_of_64_bit_region_. As a result, the addresses of vtable candidates stored in scratch_data were miscalculated, preventing proper deduplication.
Solution
This fix includes length_of_64_bit_region_ in the offset calculation for vtables in 64-bit mode. This ensures correct vtable candidate addresses, enabling deduplication and restoring efficient serialization for large buffers.
Impact
- Restores vtable deduplication for buffers >2GB with 64-bit offsets.
- Significantly improves serialization performance and reduces buffer size for large data sets.
- Maintains full backward compatibility with 32-bit buffers.
Verification
- Serialization benchmarks with large (2GB+) buffers now show linear scaling and reduced buffer size.