Open.url and a file filter with a bunch of files

[karikera]

I tried to download and unzip a bunch of files with a file filter.

    const archive = await unzipper.Open.file(zipfile);
    for (const file of archive.files)
    {
        if (file.type == 'Directory') continue;
        const filepath = ~~~~
        const exists = fs.existsSync(filepath);
        if (exists) continue;
        files.push(file);
    }
   await kindOfConcurrencyUnzipFunction(files);

file count is over 1000. and unzipper sends the request per file. pretty insane. the web server blocks requests by too many requests.

how can I make it as one request?

[warerebel]

Can't you just do:

http.get("file-url", (res) => {
    const zip = res.pipe(unzipper.Parse({forceStream: true}));
    for await (const entry of zip) {
        // etc etc

    }
});

[karikera]

@warerebel It seems better. I wasn't used to piping. thanks.