Paddy Horan

Results 10 comments of Paddy Horan

@jorgecarleitao I thought the overall aim of `arrow2` was to upstream it to the main arrow repo. Don't you think that publishing this crate and having people rely on it...

> Arguing / pressuring that I should not release a Rust implementation of the Arrow specification outside of the Apache Arrow project at the expense of splitting the community is...

> @paddyhoran if this happens would it make sense to use ndarray-stats for Arrow compute kernels? Yes, perhaps. We started to implement some compute kernels in Arrow and using SIMD,etc....

Hi @jturner314, Thanks for the detailed response. > creating an `ArrayBase` from an Arrow tensor should be possible as a zero-copy operation in the case where the strides are a...

One key question is in regard to how `ndarray` represents `null` values, i.e. `NaN`, etc. I'm assuming you use a sentinal value?

> In other words, the allocation is 64-byte aligned, but the first element of the array `unaligned` is not 64-byte aligned, which I understand to be a requirement for (zero-copy)...

Yep, makes total sense. Thanks @jturner314

> I've been reading the Buffer code -- from docs.rs/arrow -- just to understand a bit, and I can't help but mention the various questions I have. All after a...

> Integrating with Arrow seems very promising. The only "disappointment" is that Arrow focuses on columnar data, and its multidimensional type - Tensor - does not have much features or...

Thanks again for your review and input @bluss. I opened [ARROW-8480](https://issues.apache.org/jira/browse/ARROW-8480) to track the remaining issue you found.