velox
velox copied to clipboard
facebookincubator
Support merging encoded Vectors when deserializing
Summary: Exchange has logic to take a vector of pages and deserialize them all into a single RowVector.
This can break when encodings are preserved in the serialized data. Specifically when reading flat data, we used to simply resize the column Vector and deserialize directly into it since that Vector was always flat. That Vector can now be constant or dictionary encoded.
The fix is simply to check if the Vector is flat and call ensureWritable to flatten it if not.
Note that other code paths (deserializing constants, dictionaries, all null columns) already deserialize into a temporary vector, and ensureWritable is called on the output Vector before copying into it (unless this is the first page, in which case the temporary Vector is simply returned).
Differential Revision: D53740784
Deploy Preview for meta-velox canceled.
| Name | Link |
|---|---|
| Latest commit | ed348b4e1d587fc57d12c32f87ab2256fe2ec8e8 |
| Latest deploy log | https://app.netlify.com/sites/meta-velox/deploys/663d25fce217f9000847867a |
![netlify[bot] avatar](https://avatars.githubusercontent.com/in/13473?v=4 "Published by a pub.dev verified publisher"){kind=small}
