capnproto-rust icon indicating copy to clipboard operation
capnproto-rust copied to clipboard

Capnp message growing unbounded when reusing Builder to save on allocation

Open gtsui opened this issue 1 year ago • 1 comments

In my code below, I want to reuse my capnp message object because I dont want to have to allocate new memory on every cycle of my loop. However, when I run it this way, the size of my message object seems to grow and grow.

I'm sure there must be some way to have capnp write at the beginning of my message rather than appending to it, or clear it, but I'm not sure how? Am I missing something simple?

fn main() -> Result<(), anyhow::Error> {
    let mut message = capnp::message::Builder::new(HeapAllocator::new().first_segment_words(128));
    let mut buffer: Vec<u8> = Vec::with_capacity(1024);
    for i in 0..100 {
        buffer.clear();
        let mut builder = message.init_root::<book_capnp::book::Builder>();
        builder.set_author("Mark Twain");
        builder.set_title("Huckleberry Finn");
        builder.set_pages(400);
        capnp::serialize::write_message(&mut buffer, &message)?;
        println!("{:?}", buffer.len());
        println!("{:?}", message.size_in_words());
        println!("=====");
    }
    Ok(())
}

Output:

80
9
=====
144
17
=====
//...
7104
886
=====

gtsui avatar Oct 07 '24 18:10 gtsui

You can reuse the first segment of your message builder via ScratchSpaceHeapAllocator:

https://github.com/capnproto/capnproto-rust/blob/9bf527e73e9f6a5cfb4d5cd01dc72215a9b83227/capnp/src/message.rs#L799-L815

dwrensha avatar Oct 07 '24 20:10 dwrensha