Minimize buffering w/ v1.1 Binary writer?

[barries]

I've been digging through the 1.1 rc10 reference docs to figure out how to configure it to minimized buffering--for example by using delimited sequences. I'm afraid I haven't figured it out, so... how can I configure the v1.1 writer to minimize buffering and other overhead when using both the imperative and Serde serialization APIs?

Thanks!

For context, the prototype so far is:

    let writer = IonWriter::new(
        v1_1::Binary,
        File::create(log_file_path.clone())?,
    ).expect("BinaryWriter::new() failed");
    ... store writer in self.ion_writer ....

   
    pub fn write_entry(
        &mut self,
        module_path:   &str,
        event_kind:    EventKind,
        write_content: impl FnOnce(&mut Log),
    ) {
        {
            use chrono::Utc;

            let now = Utc::now().naive_utc();
            let event_timestamp = ion_rs::Timestamp::from_datetime(now, ion_rs::TimestampPrecision::Second);
            let _ = self.ion_writer.write(event_timestamp);
            let _ = self.ion_writer.write_symbol(format!("{:?}", event_kind).as_str());
            write_content(self);    // <<== usually a closure that calls write_value(...)
        }
        let _ = self.ion_writer.flush();
    }

    pub fn write_value(&mut self, value: &impl serde::Serialize) {
        use ion_rs::{
            SequenceWriter,
            serde::ser::ValueSerializer,
        };

        let is_human_readable = false;
        let serializer = ValueSerializer::new(
            self.ion_writer.value_writer(),
            is_human_readable
        );
        let _ = value.serialize(serializer);
    }

[zslayton]

You can call value_writer().with_delimited_containers() to skip buffering for containers. Setting that also sets it for any nested containers (unless you set it back for those again).

Here's the type that holds all of the currently available options: https://github.com/amazon-ion/ion-rust/blob/4e8739f75b737458a88cfea95b4972b3bf94313d/src/lazy/encoder/value_writer_config.rs#L1-L15

Each of those enums are defined just below it. Let me know if I can answer more questions.

[barries]

Thanks, @zslayton.

That's not present on ApplicationValueWriter<>, should it be, or am I using the wrong API somehow?

type IonWriter = Writer<v1_1::Binary, File>;

pub struct Log {
    ion_writer: IonWriter,
}
...
error[E0599]: no method named `with_delimited_containers` found for struct `ion_rs::lazy::encoder::writer::ApplicationValueWriter` in the current scope                                                                                        --> src/uilog.rs:418:67
    |
418 |             let mut entry_writer = self.ion_writer.value_writer().with_delimited_containers();
    |                                                                   ^^^^^^^^^^^^^^^^^^^^^^^^^ method not found in `ApplicationValueWriter<'_, BinaryValueWriter_1_1<'_, '_>>`

[zslayton]

Hmm, it should be there. The source code for those methods is here:

https://github.com/amazon-ion/ion-rust/blob/df7ba6c21b24bb2e4993ef92ca15cbbbff325747/src/lazy/encoder/writer.rs#L243-L267

Note that it's only implemented for ApplicationValueWriter<'_, BinaryValueWriter_1_1<'_, '_>> since binary 1.1 is the only encoding that has opt-in delimited containers.

[barries]

When I click on that link, I see this

When I look in my ion-rust main branch a few commits after .rc10, I only find it here:

impl BinaryValueWriter_1_1<'_, '_> {
    ...
    
    pub fn with_delimited_containers(mut self) -> Self {
        self.value_writer_config = self.value_writer_config.with_delimited_containers();
        self
    }

What am I doing wrong?