awkward
awkward copied to clipboard
List of Empty Objects is not supported
Version of Awkward Array
1.8.0
Description and code to reproduce
>>> import awkward as ak
>>> ak.to_arrow([{},{},{}])
ValueError Traceback (most recent call last)
Input In [12], in <cell line: 1>()
----> 1 ak.to_arrow([{},{},{}])
File ~/miniconda/envs/mongo-arrow_env/lib/python3.9/site-packages/awkward/operations/convert.py:2507, in to_arrow(array, list_to32, string_to32, bytestring_to32, allow_tensor)
2501 else:
2502 raise TypeError(
2503 f"unrecognized array type: {layout!r}"
2504 + ak._util.exception_suffix(__file__)
2505 )
-> 2507 return recurse(layout, None, False)
File ~/miniconda/envs/mongo-arrow_env/lib/python3.9/site-packages/awkward/operations/convert.py:2247, in to_arrow.<locals>.recurse(layout, mask, is_option)
2242 elif isinstance(layout, ak.layout.RecordArray):
2243 values = [
2244 recurse(x[: len(layout)], mask, is_option) for x in layout.contents
2245 ]
-> 2247 min_list_len = min(map(len, values))
2249 types = pyarrow.struct(
2250 [
2251 pyarrow.field(layout.key(i), values[i].type).with_nullable(
(...)
2255 ]
2256 )
2258 if mask is not None:
ValueError: min() arg is an empty sequence
It works properly with pyarrow:
>>> import pyarrow as pa
>>> pa.array([{},{},{}]).tolist()
[{}, {}, {}]
This looks like a bug to me. We don't have a fallback for RecordArrays with no child layouts.
This bug is not reproducible in our v2 release, which is being developed in main and available in the latest pre-releases under the awkward._v2 submodule.
If you need to be able to do this, then you might benefit from trying out the v2 API. It's identical (or near-enough) at the high-level to v1, and I believe that we intend on making v2 support operations on v1 arrays (certainly this currently works in our pre-releases). You can try this out with ak._v2.to_arrow(my_existing_v1_array).
Thanks @agoose77!