arrow2 icon indicating copy to clipboard operation
arrow2 copied to clipboard

UnionArray type with optional type_ids doesn't correctly round-trip on ffi

Open jondo2010 opened this issue 2 years ago • 2 comments

The UnionArray type in arrow2 allows implicitly defining the type IDs by leaving the 2nd value in the tuple None. This causes problems when round-tripping with the Arrow ffi.

Here's a minimal example that breaks the existing unit test in src/ffi/schema.rs: https://github.com/rerun-io/arrow2/commit/3a7e80922bed97b93e58b6fa47415ffdcc9686e0

Specifically, the FFI spec doesn't allow implicitly defining the type IDs, and so when serializing arrow2->ffi they get expliclitly defined (again in schema.rs), and when returning ffi->arrow2, the equality between types fails.

jondo2010 avatar Jan 17 '23 10:01 jondo2010