parquet-tools icon indicating copy to clipboard operation
parquet-tools copied to clipboard

Invalid concatenation for multiple parquet files with different columns structure

Open ktrueda opened this issue 6 months ago • 0 comments

If users specify multiple parquet files which has different column structure, the outcome is concat with column direction (not row direction).

$ poetry run parquet-tools csv ./tests/test1.parquet ./tests/test0.parquet
one,two,three,a,b,c,d
-1.0,foo,True,,,,
,bar,False,,,,
2.5,baz,True,,,,

ktrueda avatar Jan 02 '24 11:01 ktrueda