unzip-stream icon indicating copy to clipboard operation
unzip-stream copied to clipboard

How many entries?

Open kldavis4 opened this issue 7 years ago • 1 comments

I wrote a function like this:

function extractZipEntriesFromBuffer (buffer, targets) {
  return new Promise((resolve, reject) => {
    const records = {}
    const unzipParser = unzip.Parse({debug: false})
    const stream = new PassThrough()
    stream.end(buffer)
    stream
      .pipe(unzipParser)
      .pipe(Transform({
        objectMode: true,
        transform: async function (entry, e, cb) {
          const {path, type, size} = entry
          if (targets.includes(path)) {
            records[path] = await streamToString(entry, cb)
            if (Object.keys(records).length === targets.length) {
              resolve(records)
            }
          } else {
            entry.autodrain()
            cb()
          }
        }
      }))
  })
}

It takes a buffer with the zip data. I am currently passing in a list of target files that I want extracted and when I have extracted those, I resolve the promise. I'd like to just resolve the promise when all entries have been received. I tried an on close event, but then it fires before the entry events even occur. The problem right now is if the caller happens to pass in a file entry that doesn't exist, then the promise doesn't resolve.

So the question is how do I know how many entries there are or when I finished receiving them all?

kldavis4 avatar Oct 09 '18 15:10 kldavis4

The Parse stream should always emit all entries before it ends, so I'd suggest pushing the promises inside the transform to an array and connecting to the end signal, when that is invoked, use Promise.all.

Something along:

function extractZipEntriesFromBuffer (buffer, targets) {
  return new Promise((resolve, reject) => {
    const promises = []
    const unzipParser = unzip.Parse()
    unzipParser.on('end', () => {
      await Promise.all(promises);
      if (promises.length === targets.length) {
        resolve()
      } else {
        reject(new Error('extract mismatch'))
      }
    })
    const stream = new PassThrough()
    stream
      .pipe(unzipParser)
      .pipe(Transform({
        objectMode: true,
        transform: function (entry, e, cb) {
          const {path, type, size} = entry
          if (targets.includes(path)) {
            let p = streamToString(entry, cb)
            promises.push(p)
          } else {
            entry.autodrain()
            cb()
          }
        }
      }))
    stream.end(buffer)
  })
}

mhr3 avatar Oct 12 '18 09:10 mhr3