Multistream Zlib archives?

[kallisti5]

I'm working on decompressing a heap of zlib compressed streams. Each one has a defined chunk size. The first chunk is always valid, but the second chunk is always invalid.

Is there an easy way to expand multi-stream Zlib blobs?

I'm attempting to manually leverage zlib on each chunk.. I don't think a reset is needed since i'm controlling the ZlibDecoder invocation... however the data seemingly isn't decompressed as the input is identical to the output.

read_to_end seems to read to the end of the zlib stream instead of the end of the file per the documentation (maybe a mistake?)

use flate2::read::ZlibDecoder;
.
.
    /// Inflate the heap section of a hpkg for later processing
    fn inflate_heap(&mut self) -> Result<usize, Box<dyn error::Error>> {
        let chunks = self.heap_chunk_count()?;
        let header = self.header.as_ref().unwrap();
        let filename = self.filename.as_ref().unwrap();

        println!("Heap chunk size: {}", header.heap_chunk_size);
        println!("Heap chunk count: {}", chunks);
        println!("Heap compressed size: {}", header.heap_size_compressed);
        println!("Heap uncompressed size: {}", header.heap_size_uncompressed);
        println!("Heap compression: {}", header.heap_compression);

        // Each chunk is compressed individually so each represents a separate zlib stream.
        let mut pos: u64 = 0;
        while pos < header.heap_size_compressed {
            println!("Read {} - {}...", pos, pos + header.heap_chunk_size as u64);
            let mut f = File::open(filename)?;
            &f.seek(SeekFrom::Start(header.header_size as u64 + pos))?;

            let mut reader: Box<dyn Read> = match header.heap_compression {
                0 => Box::new(f),
                1 => Box::new(ZlibDecoder::new(f)),
                _ => return Err(From::from(format!("Unknown hpkg heap compression: {}", header.heap_compression)))
            };
            let len = reader.read_to_end(&mut self.heap_data)? as u64;
            pos = pos + len + 1;

            println!("Uncompressed heap: {}", self.heap_data.len());
        }
        //println!("Heap:  Decompressed Length: {} vs header length: {}", heap_pos, header.heap_size_uncompressed);
        Ok(0)
    }

$ ./target/debug/examples/dump_hpkg sample/ctags_source-5.8-5-source.hpkg 
Heap chunk size: 65536
Heap chunk count: 31
Heap compressed size: 501432
Heap uncompressed size: 1988947
Heap compression: 1
Read 0 - 65536...
Uncompressed heap: 65536
Read 65537 - 131073...
ERROR: Custom { kind: InvalidInput, error: "corrupt deflate stream" }
thread 'main' panicked at 'assertion failed: false', examples/dump_hpkg.rs:17:13
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[drtconway]

I had the same problem, and this solution worked for me.

https://github.com/rust-lang/flate2-rs/issues/339