torcharrow
torcharrow copied to clipboard
`to_arrow` doesn't work correctly on List(Struct) column
trafficstars
import torcharrow as ta
import torcharrow.dtypes as dt
a = ta.Column(
[
[(1, 1.1)],
[(2, 2.2), (3, 3.3)],
[],
[(4, 4.4), (5, None)],
[(6, 6.6)],
],
dtype=dt.List(
dt.Struct(
[
dt.Field("i", dt.Int64()),
dt.Field("f", dt.Float32(nullable=True)),
]
)
))
a.to_arrow()
Output :
>>> a.to_arrow()
<pyarrow.lib.ListArray object at 0x7fcc78b49520>
[
[
[
1,
1.100000023841858
]
],
[
[
2,
2.200000047683716
],
[
3,
3.299999952316284
]
],
[],
[
[
4,
4.400000095367432
],
[
5,
null
]
],
[
[
6,
6.599999904632568
]
]
]
Type becomes List(List)