binread icon indicating copy to clipboard operation
binread copied to clipboard

Relative Offsets

Open Waelwindows opened this issue 3 years ago • 6 comments

I have scoured through the documentation and there doesn't seem to be a way to handle offsets relative to the start of a struct. Looking at the implementation in file_ptr.rs it seems trivial to implement.

One could imagine it looking like

#[derive(BinRead)]
#[br(offsets=relative)]
struct Foo {
    bar: FilePtr32<NullString>
}

where the bar.offset would be relative to the position of Foo in the stream.

I'm writing this as an issue mostly to discuss any potential alternatives to this proposed design like the one in #13.

Waelwindows avatar Jan 10 '21 02:01 Waelwindows

I'd totally be willing to accept/help with/maybe work on a PR for something like this, although not sure I 100% agree with your design. First off it's pretty all or nothing, if you have both relative and absolute pointers in a struct the design kinda is hard to use at all?

Maybe something like...

#[derive(BinRead)]
struct Foo {
    #[br(relative)]
    bar: FilePtr32<NullString>,
}

(Probably also allow relative = $expr for runtime configuration)

The implementation likely involving iterating over all fields, and if a field uses this it creates a variable at the start of the implementation to store the position of the start of the struct, then passes it as the offset field (see the offset attribute)

jam1garner avatar Jan 10 '21 03:01 jam1garner

I really need to get the code i wrote up at some point. >_>

@jam1garner I largely agree with you. your solution sounds good, but you could also just have the user insert that type/value into the struct themselves. I suppose it'd still be nice syntax sugar.

kitlith avatar Jan 10 '21 07:01 kitlith

For anybody finding themselves in this position, here's a workaround

fn relative_offset<R, BR, A>(reader: &mut R, ro: &ReadOptions, args: A) -> BinResult<BR>
where
    R: Read + Seek,
    BR: BinRead<Args=A>,
{
    let pos = reader.seek(SeekFrom::Current(0))?;
    let mut ro = *ro;
    ro.offset = pos - ro.offset;
    BR::read_options(reader, &ro, args)
}

to be used like this

#[derive(BinRead)]
#[br(magic= 0x10000)]
struct Object {
    #[br(pad_before=4)]
    bounds: BoundingSphere,
    #[br(parse_with=relative_offset, offset=6*4)]
    meshes: FileVec<PosValue<Mesh>>,
    #[br(parse_with=relative_offset, offset=7*4)]
    materials: FileVec<PosValue<Material>>,
}

as you can see, this is hacky as you have to specify when the particular field starts in a struct in bytes as an offset attribute

Waelwindows avatar Jan 11 '21 00:01 Waelwindows

I think this is a version that's closer to jam's proposed solution, just without the macro syntax magic @Waelwindows

pub struct CurPos(pub u64);

impl BinRead for CurPos {
    type Args = ();

    fn read_options<R: Read + Seek>(reader: &mut R, ro: &ReadOptions, args: Self::Args) -> BinResult<Self> {
        Ok(CurPos(reader.seek(io::SeekFrom::Current(0))?))
    }
}

setting up with the same struct you used as an example:

#[derive_binread]
#[br(magic= 0x10000)]
struct Object {
    #[br(temp)]
    starting_position: CurPos,
    #[br(pad_before=4)]
    bounds: BoundingSphere,
    #[br(offset=starting_position.0 - 4)]
    meshes: FileVec<PosValue<Mesh>>,
    #[br(offset=starting_position.0 - 4)]
    materials: FileVec<PosValue<Material>>,
}

we could also do the equivalent with a parse_with function and a u64 directly instead of a wrapper type, but shrug

kitlith avatar Jan 11 '21 01:01 kitlith

This is a great solution, however keep in mind that the magic isn't taken into account, so you'll have to account for that

Waelwindows avatar Jan 11 '21 13:01 Waelwindows

true enough, but at least you only have to subtract the size of the magic, and if you had a file format where offsets are relative to some other part of the struct, you can place the type directly there. I'll add those offsets to the example... though it's not clear exactly what the type of the magic is. guessing u32.

kitlith avatar Jan 11 '21 22:01 kitlith