cxx icon indicating copy to clipboard operation
cxx copied to clipboard

Could `rust::Vec::set_len` be public API?

Open benesch opened this issue 3 years ago • 3 comments

I'm writing some code in C++ that reserves space in a rust::Vec, initializes the new chunk of memory, and then wants to call rust::Vec::set_len to indicate that the new elements are now valid. That's obviously "unsafe" from Rust's perspective but doesn't seem particularly more dangerous than any other C++ code. But rust::Vec::set_len is currently private, presumably for a reason! Thoughts on making it public? Perhaps with an unsafe prefix, as in unsafe_set_len?

Note: not blocked on this, since I've got this workaround:

#[cxx::bridge]
mod ffi {
    extern "Rust" {
        unsafe fn vec_u8_set_len(v: &mut Vec<u8>, new_len: usize);
    }
}

unsafe fn vec_u8_set_len(v: &mut Vec<u8>, new_len: usize) {
    v.set_len(new_len)
}

benesch avatar Dec 27 '21 18:12 benesch

How is this API done in std::vector or other C++ vector libraries?

dtolnay avatar Dec 27 '21 19:12 dtolnay

As best as I can tell, the answer for std::vector (and std::string) is that there's no built-in way to do this. Various resources on the matter:

  • folly has a gross set of hacks that allow you to reach in and call the non-public set_size method: https://github.com/facebook/folly/blob/5b5e5909324ce9898509fbb61a8d1077a5d49924/folly/memory/UninitializedMemoryHacks.h#L42-L77
  • Google alleges to have similar code internally, but they remove it before it reaches the open source world: https://github.com/protocolbuffers/protobuf/blob/8c29dc2c4d55bb620bfe15ca6b74f8097cbb006d/src/google/protobuf/stubs/stl_util.h#L45-L53
  • boost::container::vector has a resize overload that default-initializes any new members rather than value-initializing them. (See https://www.boost.org/doc/libs/1_78_0/doc/html/container/extended_functionality.html#container.extended_functionality.default_initialialization.) This doesn't help for classes, but for primitive types it means resize creates uninitialized elements rather than zero-initialized elements.
  • You can achieve something similar to boost::vector with std::vector by using a custom allocator: https://stackoverflow.com/questions/15097783/value-initialized-objects-in-c11-and-stdvector-constructor/15119665#15119665

Caveat: It's been a long time since I've seriously written C++. I'm certainly not an expert on this.

benesch avatar Dec 27 '21 22:12 benesch

Here's a StackOverflow that addresses this directly: https://stackoverflow.com/a/61450370.

The tl;dr is the idiom of reserving space in a std::vector, writing to it, and then hackily increasing the size of the vector is technically undefined behavior, but de facto no one cares? What a mess.

benesch avatar Dec 27 '21 22:12 benesch