neon-bindings
Detachable `ArrayBuffer`
Neon allows allocating ArrayBuffer with bytes with ArrayBuffer::external. Buffers created this way can be detached with napi_detach_array_buffer.
Unfortunately, Node-API does not provide a way to get the finalize_hint back out of the external ArrayBuffer. Since ArrayBuffer::external is generic, there is no way to safely drop the data when done.
Neon should provide a special detachable array buffer type that always uses an exactly sized Vec<u8>. Since Neon will provide multiple buffer types that can be detached (according to Node-API), detachable buffers will need to be tagged with napi_type_tag_object, locking the feature to Node-API 8+.
Unlike dereferencing from a Root, this API will provide a completely safe way to transfer bytes back and forth between JavaScript and Neon without copying data.
- Create
Vec<u8>in Rust - Transfer to JavaScript with
JsArrayBuffer::detachable(&mut cx, data) - Take data back with
let data = buf.detach(&mut cx).or_throw(&mut cx)? - Re-attach again with
JsArrayBuffer::detachable(&mut cx, data)
There might be a way this could be implemented without a special detachable ArrayBuffer type. Neon could maintain a lookup of std::any::TypeId to napi_type_tag.
Types that implement trait Detachable could be reconstructed from this information.
Unfortunately, I think this might be problematic for types that need more than a pointer and length to reconstructed (e.g., Vec<_> needs a capacity). A possible solution is to maintain a lookup of data address to struct addresses. This would require data addresses to always point to exactly one struct which I think should be guaranteed by AsMut. We could have an assert that guards this invariant.
FWIW, Box<[_]> is a perfectly-good non-resizable Vec. (They even interconvert, with Vec -> Box dropping excess capacity and Box -> Vec assuming no excess capacity.)
@jrose-signal That's a good idea. It might be better to use impl Into<Box<[u8]>> instead of explicitly taking a Vec and resizing it.
Do you think having creating an ArrayBuffer from AsMut<[u8]> is useful? It simplifies things if externals are only created with a single type.
Hm. The mutability requirement rules out a lot of common cases, including places I'd like to be to zero-copy. But there are still things like bytes::BytesMut that won't be viable. Still, maybe that's fine. (I feel like the use case for this is pretty limited, honestly, because it's hard to guarantee when detaching that the buffer being handed to you came from Rust originally.)