binread icon indicating copy to clipboard operation
binread copied to clipboard

Terminate/TakeUntil attribute for dynamically sized vecs

Open Waelwindows opened this issue 3 years ago • 1 comments

In the binary formats i'm parsing, there are Vecs that don't have an explicit length stored in the file with them. One common method of storing such Vecs is by parsing till a sentinel value is encountered. such as a nullptr or -1

So far I've been using a custom parser, and that works well, until you need custom Args or your field is a Vec wrapped up in something like a FilePtr

fn terminate<R, BR, F>(reader: &mut R, ro: &ReadOptions, args: (F,)) -> BinResult<Vec<BR>>
where
    R: Read + Seek,
    BR: BinRead<Args=()>,
    F: FnMut(&BR) -> bool,
{
    let mut vec = vec![];
    let mut f = args.0;
    let args = ();
    loop {
        let mut br = BR::read_options(reader, ro, args)?;
        if f(&br) {
            break;
        }
        br.after_parse(reader, ro, args)?;
        vec.push(br);
    }
    Ok(vec)
}

Keep note that the condition is checked before BinRead::after_parse to allow you to check for nullptrs and the such. As doing them after would cause errors if the offset isn't valid.

Here's an example of my custom parser in use

#[derive(Debug, BinRead)]
struct OsageSiblingInfos {
    #[br(parse_with=terminate, args(|x: &OsageSiblingInfo| x.bone_name.ptr == 0))]
    infos: Vec<OsageSiblingInfo>,
}

#[derive(Debug, BinRead)]
struct OsageSiblingInfo {
    bone_name: FilePtr32<NullString>,
    name: FilePtr32<NullString>,
    distance: f32,
}

I'm hoping the attribute could work something like

#[derive(Debug, BinRead)]
struct OsageSiblingInfos {
    #[br(terminate=|x| x.bone_name.ptr == 0)]
    infos: FilePtr32<Vec<OsageSiblingInfo>>,
}

#[derive(Debug, BinRead)]
struct OsageSiblingInfo {
    bone_name: FilePtr32<NullString>,
    name: FilePtr32<NullString>,
    distance: f32,
}

Note how the closure argument is inferred instead of explicitly set in the previous example and how the field can be a Vec wrapped in something.

Waelwindows avatar Jan 13 '21 02:01 Waelwindows

given that count is handled as an attribute? yeah, this probably belongs as an attribute too.

The location of the condition makes sense.

Until the attribute has been written...:

I don't particularly see how arguments make things much harder, esp since #13 is implemented. (but not documented, thanks for reminding me ._. https://github.com/jam1garner/binread_derive/pull/6 )

for FilePtr<, Vec<>> situations, until an attribute is added, I'd probably try to create a generic "TerminatedVec" type that you use in place of Vec and pass the correct arguments to. Let me know if you'd like help setting that up.

kitlith avatar Jan 16 '21 07:01 kitlith