json icon indicating copy to clipboard operation
json copied to clipboard

StreamDeserializer with offsets iterator

Open wyfo opened this issue 2 years ago • 1 comments

StreamDeserializer::bytes_offset can be used to get the slice of a deserialized JSON, however, it can't be used with the iterator interface, and iteration has to be done manually using Iterator::next (see #303).

In one of my project, I use the following code:

use serde_json::StreamDeserializer;
use std::ops::Range;

pub struct StreamDeserializerWithOffsets<'de, R, T>(pub StreamDeserializer<'de, R, T>);

impl<'de, R, T> Iterator for StreamDeserializerWithOffsets<'de, R, T>
where
    R: serde_json::de::Read<'de>,
    T: serde::de::Deserialize<'de>,
{
    type Item = serde_json::Result<(T, Range<usize>)>;
    fn next(&mut self) -> Option<Self::Item> {
        let start_offset = self.0.byte_offset();
        self.0
            .next()
            .map(|res| res.map(|v| (v, start_offset..self.0.byte_offset())))
    }
}

Would this feature be interesting enough to be integrated into the library (ofc i'm volounteer for the PR)? In my case, I have to use a new type, but it could be a method like StreamDeserializer::with_offsets (or StreamDeserializer::with_slice, replacing Range<usize> by &[u8]/&str).

Also, because of #70, maybe the iterator item should be (serde_json::Result<T>, Range<usize>) instead of `serde_json::Result<(T, Range)>

wyfo avatar Sep 27 '22 13:09 wyfo

Any updates to this? The json standard library for Go supports something similar: https://pkg.go.dev/encoding/json#Decoder.InputOffset

friendlymatthew avatar Jan 10 '24 00:01 friendlymatthew