archiver pipe into PassThrough Stream: without "data" event handler, stream never finishes

[DaCao]

I have the following nodejs code, where I try to append readable streams to node-archiver and pipe the archiver to a nodejs PassThrough stream:

  
  const archive = Archiver('zip', {
    zlib: { level: zlib.constants.Z_BEST_SPEED },
    highWaterMark: 10 * 1024 * 1024
  });
  archive.on('error', (error) => {
    logger.error(`archive on error: ${error.name} ${error.code} ${error.message} ${error.path} ${error.stack}`);
    throw new Error(`${error.name} ${error.code} ${error.message} ${error.path} ${error.stack}`);
  });


  const streamPassThrough = new Stream.PassThrough();


  logger.info(`check 5`)

  await new Promise((resolve, reject) => {
    logger.info("Starting upload of the output Files Zip Archive");
    
    logger.info(`check 5.1`)
    s3FilesDownloadSteams.forEach((item) => {
      logger.info(`archive.append: item.fileName: ${item.fileName}`)
      archive.append(item.stream, { name: item.fileName })
    });

    logger.info(`check 5.2`);
    archive.pipe(streamPassThrough);
    logger.info(`check 5.3`)
    // streamPassThrough.on('data', (chunk) => {
    //   console.log(`Received ${chunk.length} bytes of data.`);
    // });
    logger.info(`check 5.4`)
    streamPassThrough.on('close', resolve);
    logger.info(`check 5.5`)
    streamPassThrough.on('end', resolve);
    logger.info(`check 5.6`)
    streamPassThrough.on('error', reject);
    logger.info(`check 5.7`)
    archive.finalize();
    logger.info(`check 5.8`)
  }).catch((error) => {
    logger.error(`Stream flow error: ${error.name} ${error.code} ${error.message} ${error.path} ${error.stack}`);
  });

  logger.info(`check 6`)

However, with the streamPassThrough event handler for 'data' commented out:

    // streamPassThrough.on('data', (chunk) => {
    //   console.log(`Received ${chunk.length} bytes of data.`);
    // });

The code never leave the Promise, check 5.1 to check 5.8 gets printed but check 6 never gets printed.

The weird thing is that if I un-comment out the streamPassThrough event handler for 'data', like this:

    logger.info(`check 5.3`)
    streamPassThrough.on('data', (chunk) => {
       console.log(`Received ${chunk.length} bytes of data.`);
    });
    logger.info(`check 5.4`)

Then it works fine and check 6 is printed.

What is going on here? Why does the streamPassThrough event handler for 'data' affects the code at all?

[cor1]

im seeing the same thing, any idea?

[AmeyMore98]

@DaCao It could be related to this. Not sure though.

[mch]

It's because the streamPassThrough starts paused, so the close and end events are never emitted and the promise never resolves. See Node.js Streams, two reading modes. When you attach the data handler, it unpauses and eventually the close and end events are emitted, resolving the promise.