Archiver.append() does not work on NodeJS v16.2

[Ylianst]

I noticed that calling .append() with a buffer does not work on NodeJS v16. I tested it on various v6 to v15 versions and it works, but in NodeJS v16.2 the .append call will pack the data correctly but put a 0 as the file length which results of a corrupt archive.

var archive = require('archiver')('zip', { level: 9 }); // Sets the compression method.
...
archive.append(data, { name: 'filename' }); // Does not work on NodeJS v16, results 0 size in zip header.

A workaround is to turn the Buffer into a stream and pass it in:

var archive = require('archiver')('zip', { level: 9 }); // Sets the compression method.
...
const ReadableStream = require('stream').Readable;
const zipInputStream = new ReadableStream();
zipInputStream.push(data);
zipInputStream.push(null);
archive.append(zipInputStream, { name: 'filename' });

This workaround works on all NodeJS v11 to v16.2 that I have tested.

[ctalkington]

hum did node change the buffer api again? I haven't read up on 16 but if it's not a documented breaking change it may be worth seeing if it's a bug as we've caught such in past and they added tests to confirm compatibility

FWIW our tests pass on v16 so might need more details about the data buffer, os/env etc to try and replicate issue

[peterpme]

Hi there!

We're also facing this issue at Draftbit. I upgraded the archiver package to see if that helps. I'm wondering if ya'll have made any progress / if this issue is stale? Otherwise that fix should work for us :)

Thank you!

[peterpme]

It looks like the tests run against v16.7 and not v16.12/13 so this could still be an issue.

[peterpme]

Following up, explicitly from our project's POV:

• Using v16.13 doesn't work
• No other environment changes were made.
• Using docker: node:16.13-alpine
• Downgrading to node 16.7 worked (including node:16.7-alpine docker)

I looked through the nodejs changelog and noticed several references to buffer changes being made. We're getting near the infamous Friday Deployment Cutoff hour so in our case we solved this by downgrading temporarily.

@ctalkington if I can help with any more information, please let me know. We'll be alright on v16.7 in the mean time. No pressure. Thank you!

EDIT: 16.12 does in fact work. 16.13 doesn't. Just tried it again to make sure.

[ghost]

@ctalkington I believe this issue is related to the upstream package crc32-stream. In version 4.0.2 there was a bug-fix. See https://github.com/archiverjs/node-crc32-stream/releases/tag/4.0.2.

archiver version 5.0.3 references compress-commons version 4.1.0 and this in turn references crc32-stream version 4.0.1. If compress-commons's dependencies are updated and in turn archivers's I believe this bug will be fixed.

In the meantime we're resolving via package resolutions to "crc32-stream": "^4.0.2",. As an FYI the readable stream workaround did the trick also.

[peterpme]

Good call, @matthewt-codaio !

[theaceofthespade]

Here's a simple shim of @Ylianst's solution in case anyone else needs it. This should allow you to use archive.append like normal.

` const archive = archiver("zip");

// archiver+node 16 bug workaround
// https://github.com/archiverjs/node-archiver/issues/542
const appendFunc = archive.append.bind(archive);
archive.append = (data, options) => {
  const zipInputStream = new Readable();
  zipInputStream.push(data);
  zipInputStream.push(null);

  appendFunc(zipInputStream, options);
};`

[maximelafarie]

Nothing new about an "official" fix about this issue?

[gjuchault]

The following script:

const fs = require("fs");
const stream = require("stream");

const archive = require("archiver")("zip", { level: 9 });

const buffer = Buffer.from("Hello");
const readable = new stream.Readable();
readable.push(buffer);
readable.push(null);

archive.append(buffer, { name: "buffer" });
archive.append(readable, { name: "readable" });

archive.on("error", (error) => {
  throw new Error(error);
});

archive.finalize();

const output = fs.createWriteStream(__dirname + "/example.zip");
archive.pipe(output);

Works properly with Node v16.13.2, or Node v16.18.0

Can be explained by the fact that archiver@npm:5.3.1 is using crc32-stream@npm:4.0.2

I think the issue can be closed