substrait icon indicating copy to clipboard operation
substrait copied to clipboard

Clarifying `start` and `length` in `ReadRel`

Open save-buffer opened this issue 1 year ago • 7 comments

Hi, I just came a cross an issue when running Velox on a parquet file. My plan omitted the start and length fields of the LocalFiles message which caused both to be set to 0, which meant that Velox read 0 bytes at offset 0, producing no results. I think in a huge percentage of cases people will want to read the entire file. Would it be reasonable to specify these fields as optional or to make 0 for the length mean to read the whole thing? I think technically Velox is conformant right now, but this isn't the behavior I'd expect.

save-buffer avatar Nov 04 '22 19:11 save-buffer