s3cmd
s3cmd copied to clipboard
Uploading Empty file when piping with compression
Uploading Empty file when piping with compression
tar -cf - -C "00" . | pzstd > ./test_bkp.tar.zst
[root@docker-desktop:/xbackup# ls
00 02 final latest_backup.txt test_bkp.tar.zst
root@docker-desktop:/xbackup# du ish test_bkp.tar.zst
du: cannot access 'ish': No such file or directory
2214520 test_bkp.tar.zst
root@docker-desktop:/xbackup# du -sh test_bkp.tar.zst
2.2G test_bkp.tar.zst
root@docker-desktop:/xbackup# ^C
root@docker-desktop:/xbackup# s3cmd --config=/root/.s3cfg put ./test_bkp.tar.zst s3://pl-rails-db/xbackup/test_bkp.tar.zst --encrypt --verbose
INFO: No cache file found, creating it.
INFO: Compiling list of local files...
INFO: Running stat() and reading/calculating MD5 values on 1 files, this may take some time...
INFO: Summary: 1 local files to upload
INFO: Encrypting file ./test_bkp.tar.zst to /tmp/tmpfile-Z4t4eDelPWMHo06lYV7W...
... truncated logs( here it's working correctly)
root@docker-desktop:/xbackup# ls
00 02 final latest_backup.txt test_bkp.tar.zst
root@docker-desktop:/xbackup# rm test_bkup.tar.zst
rm: cannot remove 'test_bkup.tar.zst': No such file or directory
root@docker-desktop:/xbackup# rm test_bkp.tar.zst
root@docker-desktop:/xbackup# tar -cf - -C "00" . | pzstd | tee /tmp/test_bkp.tar.zst | s3cmd --config=/root/.s3cfg put - s3://pl-rails-db/xbackup/test_bkup.tar.zst --encrypt --verbose
INFO: No cache file found, creating it.
INFO: Compiling list of local files...
INFO: Running stat() and reading/calculating MD5 values on 1 files, this may take some time...
INFO: Summary: 1 local files to upload
INFO: Encrypting file - to /tmp/tmpfile-bsWEPE5NPfYLDHBgxVg1...
upload: '/tmp/tmpfile-bsWEPE5NPfYLDHBgxVg1' -> 's3://pl-rails-db/xbackup/test_bkup.tar.zst' [1 of 1]
70 of 70 100% in 0s 109.63 B/s done
When I compress and upload separately it's getting uploaded correctly else just an empty file (70 B of data getting uploaded). Found no error in logs or by running even with verbose mode
tar -cf - -C "00" . | pzstd | s3cmd --config=/root/.s3cfg put - s3://pl-rails-db/xbackup/test_bkup.tar.zst --verbose
It's working without the --encrypt option