generic-array icon indicating copy to clipboard operation
generic-array copied to clipboard

Instantiating the array via default seems to have a very large stack footprint

Open prestwich opened this issue 5 years ago • 6 comments

May be misunderstanding something here, but it looks like the instantiation process has an unnecessarily heavy stack footprint. I expect to have a few MB of stack space available, but can't instantiate an array of size larger than ~200kB.

#[cfg(test)]
mod test {
    #[test]
    fn it_defaults() {
        // 208896
        dbg!(std::mem::size_of::<GenericArray<[u64; 32], U816>>());
        // 209152
        dbg!(std::mem::size_of::<GenericArray<[u64; 32], U817>>());
        
        // Works
        GenericArray::<[u64; 32], U816>::default();
        
        // Stack Overflow
        GenericArray::<[u64; 32], U817>::default();
    }
}

prestwich avatar Jul 30 '20 22:07 prestwich

Interesting. Anything over U806 overflows the stack on my machine. I'll look into this more.

novacrazy avatar Jul 30 '20 23:07 novacrazy

Some additional info from playing around with this: Using generic_array::transmute to turn large arrays into large GenericArrays will stack overflow, while using std::mem::transmute will work

prestwich avatar Jul 30 '20 23:07 prestwich

Yes, generic_array::transmute requires copying the data, but it's not used here.

The problem is mostly due to Rust/LLVM in the unoptimized debug mode simply not performing ANY return-value optimization, resulting in many duplicate copies on the stack as it is passed around. There isn't much I can do about that, other than providing an API to initialize a generic-array within an already existing chunk of memory, but that's rather obtuse.

Perhaps there is an issue for Rust itself regarding return-value optimizations, but for now all I can do is recommend you compile with opt-level=1 or 2 for your dev profile.

novacrazy avatar Jul 31 '20 00:07 novacrazy

sounds good, thanks!

prestwich avatar Jul 31 '20 00:07 prestwich

RFC about Return Value Optimization / Placement by Return: https://github.com/rust-lang/rfcs/pull/2884

sollyucko avatar Jul 31 '20 00:07 sollyucko