torcharrow icon indicating copy to clipboard operation
torcharrow copied to clipboard

`to_arrow` doesn't work correctly on List(Struct) column

Open wenleix opened this issue 4 years ago • 0 comments
trafficstars

import torcharrow as ta
import torcharrow.dtypes as dt

a = ta.Column(
    [
        [(1, 1.1)],
        [(2, 2.2), (3, 3.3)],
        [],
        [(4, 4.4), (5, None)],
        [(6, 6.6)],
    ],
    dtype=dt.List(
        dt.Struct(
            [
                dt.Field("i", dt.Int64()),
                dt.Field("f", dt.Float32(nullable=True)),
            ]
        )
    ))

a.to_arrow()

Output :

>>> a.to_arrow()
<pyarrow.lib.ListArray object at 0x7fcc78b49520>
[
  [
    [
      1,
      1.100000023841858
    ]
  ],
  [
    [
      2,
      2.200000047683716
    ],
    [
      3,
      3.299999952316284
    ]
  ],
  [],
  [
    [
      4,
      4.400000095367432
    ],
    [
      5,
      null
    ]
  ],
  [
    [
      6,
      6.599999904632568
    ]
  ]
]

Type becomes List(List)

wenleix avatar Nov 12 '21 21:11 wenleix