arrow-julia icon indicating copy to clipboard operation
arrow-julia copied to clipboard

Published 20 hours ago • verified publisher icon apache

Support Tables.partitions when reading "arrow file" format in addition to "stream" format

Open complyue opened this issue 2 years ago • 0 comments

I see #23 closed but only stream format got supported.

How about files in "arrow file" format, each with multiple record batches inside?

With released version I get a single Table as the result of Tables.partitions(tbl) is a ::Tuple{Arrow.Table}, however the columns there are chunked.

Can we get partition tables with contiguous column arrays via Tables.partitions?

As for workarounds before that's supported, any public API I can use to obtain contiguous column arrays per each record batches?

complyue avatar Mar 02 '22 15:03 complyue