node-firestore-backup icon indicating copy to clipboard operation
node-firestore-backup copied to clipboard

Support creating a compressed archive directly

Open lewchuk opened this issue 7 years ago • 3 comments
trafficstars

The JSON files created by backup are highly compressible, especially if documents in a collection have similar fields. For my data a basic zip got ~70% compression. It would be great if the backup supported writing directly to a compressed archive to limit intermediate disk space.

lewchuk avatar May 03 '18 20:05 lewchuk

Thinking out loud this makes sense to include in the tool vs recommending in people's workflow if it's a common usage we want to encourage. It makes complete sense to me, does anyone have a suggestion why this wouldn't?

yoiang avatar May 04 '18 02:05 yoiang

Since this is a tool used mainly in scripts, I'd say it might be better to allow piping the data to another program rather than creating a specific zip with specific properties that people would probably like to change. Please note that, if the DB size is bigger than available memory, then I'm not sure piping will work (and also zipping for that matter).

atlanteh avatar Jul 21 '18 20:07 atlanteh

Since I was just told this tool uses multiple files, I think that piping will be a bit harder, so maybe the zip should be good enough.

atlanteh avatar Jul 21 '18 21:07 atlanteh