s5cmd
s5cmd copied to clipboard
Add option to stdin to cp
Basically the other end of https://github.com/peak/s5cmd/issues/20, which resulted in s5cmd cat. It doesn't look like using s5cmd run or cp has the magic "-" option to take in a piped input from stdin.
Use cases: Pipe a tar archive (or any streamed command output) into s5cmd without needing a intermediary temp file. This also solves cases where we want to preserve timestamps, permissions, etc.
Example:
tar -cf - foo | s5cmd cp - s3://bar/foo.tar
or if using s5cmd run
tar -cf - foo | echo "cp - s3://bar/foo.tar" | s5cmd run
Second example passes bytes of tar archive to echo which ignores standard input and just passes cp - s3://bar//foo.tar to s5cmd run's standard input. I think this is not the intended behavior.
Can't support for - just be added without removing cat? taking bytes from stdin would be very handy for use in lambdas.
Agree that this would be very useful
I'm looking to use s5cmd to do database backups. Postgres can stream to stdout. Is there a trick for using s5cmd to stream that to s3, so I don't have to put it on disk first? I know s5cmd has been around awhile, so perhaps there's some CMD trick folks are using to work around this?
It would be nice to have a "pipe" command similar to https://docs.min.io/docs/minio-client-complete-guide.html#pipe s5cmd is terribly faster so we switched to this tool!
@igungor Can we achive an implementation similar to this ? https://rclone.org/commands/rclone_rcat/
Would love this feature. Just tried and it doesn't work.
dd if=/dev/nvme0n1 bs=4M | pv | s5cmd cp - s3://home/backup/pc.img
I don't have room to even compress this nvme locally