Feature Request: Unpacking Array<[A; N], D> into Array<A, D::Larger>

[gekailin]

trafficstars

In certain scenarios, it is convenient to create a new dimension by unpacking arrays. For example:

let angles = array![0.1, 0.2, 0.3];
let vectors = angles.mapv(|a| [a.cos(), a.sin()]); 
// Here, `vectors` is of type `Array1<[f64; 2]>`, but an `Array2<f64>` is preferred.

To address this, I have written an extension trait:

pub trait ArrayUnpackExt<A, D, const N: usize>
where
    D: Dimension,
{
    fn unpack(self) -> Array<A, D::Larger>;
}

impl<A, D, const N: usize> ArrayUnpackExt<A, D, N> for Array<[A; N], D>
where
    D: Dimension,
{
    fn unpack(self) -> Array<A, D::Larger> {
        assert!(mem::size_of::<[A; N]>() == mem::size_of::<A>() * N);
        let ndim = self.ndim();
        let mut sh = self.raw_dim().insert_axis(Axis(ndim));
        sh[ndim] = N;
        let mut st = D::zeros(ndim);
        self.strides().iter().enumerate().for_each(|(i, &v)| {
            st[i] = (v * (N as isize)) as usize;
        });
        let st = st.insert_axis(Axis(ndim));
        let (vec, off) = self.into_raw_vec_and_offset();
        match off {
            Some(0) => {
                let vec = {
                    let mut vec = mem::ManuallyDrop::new(vec);
                    let ptr = vec.as_mut_ptr() as *mut A;
                    let len = vec.len() * N;
                    let cap = vec.capacity() * N;
                    unsafe { Vec::from_raw_parts(ptr, len, cap) }
                };
                unsafe { Array::from_shape_vec_unchecked(sh.strides(st), vec) }
            }
            _ => unreachable!(), // Unable to handle non-zero offsets even with unsafe code
        }
    }
}

I believe this functionality would be a valuable addition to the ndarray crate, allowing users to easily transform arrays into higher dimensions when needed. I hope you consider embedding this feature into the library.

Thank you for your attention and consideration.

[gekailin]

I want to emphasize that the trait above is designed to be zero-copy, meaning it efficiently transforms the array structure without duplicating data. This ensures optimal performance and memory usage, making it an ideal solution for handling large datasets.

[akern40]

@gekailin I think this is a cool suggestion. It feels kinda like it falls under the same umbrella as #1463, and its solution feels kind of itertools-esque. It would be great if this worked for both arrays and homogeneous tuples.

A few notes / questions: Is the non-zero offset actually unreachable, or can we just not handle it? If it's the latter, I'd suggest that the method return Result rather than panicking. In addition, I don't think the trait itself needs the const-generic, just the impl.

[gekailin]

@akern40

1. Handling non-zero offsets isn't feasible outside the crate. However, within the crate, we can simplify the implementation by directly maintaining the .ptr field.
2. You're right—the trait should eliminate the const-generic parameters. Thank you for the suggestion.
3. I'm not familiar with homogeneous tuples. Could you help to complete this?

[akern40]

@gekailin sorry it's been a bit, but I would be happy to accept a method like this. When I talk about homogeneous tuples, I'm talking about things like (f32, f32) or, generically, (A, A). But turns out they are not guaranteed to be transmutable to arrays, so ignore that comment!

Would you be interested in completing this as a method on Array?

[thorn132]

This would help with numpy compatibility too, making it easy for a PyArray to borrow from an ArrayView<[T; N]> where T: numpy::Element.