Jim Pivarski

Results 620 comments of Jim Pivarski

This is what I meant by the procedure. The key thing is that we're applying the transformation to each array at a time. They're not getting broadcasted, but they're getting...

> we probably should error if the user tries to create such a file, as we reserve the root `""` field Users creating fields with empty names is not a...

In the above, where I thought that `ak.to_arrow_table` was wrong, it's because I didn't include the `._v2`. The v1 function was misinterpreting the v2 array and making a mess of...

```python >>> array.show(type=True) type: 9 * { z: { x: int64, y: float64 } } [{z: {x: 1, y: 1.1}}, {z: {x: 2, y: 2.2}}, {z: {x: 3, y: 3.3}},...

Constructing the Parquet data with pyarray alone, I can't reproduce the segfault. Without ExtensionArrays: ```python import numpy as np import pyarrow as pa import pyarrow.parquet as pq one = pa.Array.from_buffers(...

Actually, that wasn't quite the right test; this is: ```python >>> import pyarrow.parquet as pq >>> pq.ParquetFile("shallow.parquet").read_row_groups([0], ["column.one"]) pyarrow.Table column: struct child 0, one: int64 not null ---- column: [...

If I create a tmp.parquet that would segfault in `ak._v2.from_parquet`, ```python >>> ak._v2.from_parquet("tmp.parquet", columns=["z.x"]) Segmentation fault (core dumped) ``` the same does not segfault when being read directly by pyarrow...

So far, it looks like Awkward is preparing Arrow arrays in the same format as the all-pyarrow method. Though there must be some difference.

Reading back this tmp.parquet file (produced by Awkward), with pure pyarrow causes the segfault, if we register our mock ExtensionType with the name `"awkward"`: ```python >>> import pyarrow.parquet as pq...