awkward icon indicating copy to clipboard operation
awkward copied to clipboard

List of Empty Objects is not supported

Open blink1073 opened this issue 3 years ago • 2 comments

Version of Awkward Array

1.8.0

Description and code to reproduce

>>>  import awkward as ak
>>> ak.to_arrow([{},{},{}])
ValueError                                Traceback (most recent call last)
Input In [12], in <cell line: 1>()
----> 1 ak.to_arrow([{},{},{}])

File ~/miniconda/envs/mongo-arrow_env/lib/python3.9/site-packages/awkward/operations/convert.py:2507, in to_arrow(array, list_to32, string_to32, bytestring_to32, allow_tensor)
   2501     else:
   2502         raise TypeError(
   2503             f"unrecognized array type: {layout!r}"
   2504             + ak._util.exception_suffix(__file__)
   2505         )
-> 2507 return recurse(layout, None, False)

File ~/miniconda/envs/mongo-arrow_env/lib/python3.9/site-packages/awkward/operations/convert.py:2247, in to_arrow.<locals>.recurse(layout, mask, is_option)
   2242 elif isinstance(layout, ak.layout.RecordArray):
   2243     values = [
   2244         recurse(x[: len(layout)], mask, is_option) for x in layout.contents
   2245     ]
-> 2247     min_list_len = min(map(len, values))
   2249     types = pyarrow.struct(
   2250         [
   2251             pyarrow.field(layout.key(i), values[i].type).with_nullable(
   (...)
   2255         ]
   2256     )
   2258     if mask is not None:

ValueError: min() arg is an empty sequence

It works properly with pyarrow:

>>> import pyarrow as pa
>>> pa.array([{},{},{}]).tolist()
[{}, {}, {}]

blink1073 avatar Aug 08 '22 14:08 blink1073

This looks like a bug to me. We don't have a fallback for RecordArrays with no child layouts.

This bug is not reproducible in our v2 release, which is being developed in main and available in the latest pre-releases under the awkward._v2 submodule.

If you need to be able to do this, then you might benefit from trying out the v2 API. It's identical (or near-enough) at the high-level to v1, and I believe that we intend on making v2 support operations on v1 arrays (certainly this currently works in our pre-releases). You can try this out with ak._v2.to_arrow(my_existing_v1_array).

agoose77 avatar Aug 10 '22 11:08 agoose77

Thanks @agoose77!

blink1073 avatar Aug 10 '22 17:08 blink1073