s3cmd
s3cmd copied to clipboard
s3tools
profile support
I would like to see profiles supported. Where the account key / secret key pairs can be stored and referenced on the command line. I'd like to see support beyond what I observe with aws-cli so for example data can be downloaded from one s3 bucket ( key1/secret1 --requester-pays) and uploaded to a second s3 bucket (key2/secret2) with little or no local storage consumed and the MD5 checksum is calculated just once, etc.
Hi, Now I understand what is your use case. Sadly, it is a little bit more complicated than just the support for profiles. So far, remote 2 remote is performed by using a connection to a single s3 server and it is this server that will move files within "the cloud" as it is more efficient like that. What would be needed for your case is to have 2 concurrent connections with different servers for a new different remote 2 remote function.
That would be possible but so far would require a lot of work, so I don't think that it could come anytime soon.
Florent,
Thank you very much for taking the time to reply to me directly.
If s3cmd had an option for how the destination file is created, this could be a work around to the identity problem. WinZIP has this option. Normally, (I guess) the target file is created at the destination with a temporary name. Then when transfer is successful, a move or hardlink to the desired name occurs. This is a problem when the destination is a yas3fs mount to S3 storage. If the destination file can (with an supporting option) be created in one step, with the desired name, the yas3fs can mount the second AWS s3 area using different credentials.
-joe
On 2020-06-05 17:26, Florent Viard wrote:
Hi, Now I understand what is your use case. Sadly, it is a little bit more complicated than just the support for profiles. So far, remote 2 remote is performed by using a connection to a single s3 server and it is this server that will move files within "the cloud" as it is more efficient like that. What would be needed for your case is to have 2 concurrent connections with different servers for a new different remote 2 remote function.
That would be possible but so far would require a lot of work, so I don't think that it could come anytime soon.
-- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or unsubscribe [2].
Links:
[1] https://github.com/s3tools/s3cmd/issues/1110#issuecomment-639828925 [2] https://github.com/notifications/unsubscribe-auth/AEYRSU7KHHMFJCAF2YSAMZDRVFPIBANCNFSM4NQBLT3Q
I also prefer to use AWS_PROFILE instead of aws access key and secret key with tools. I also think that people kind of expect aws related tooling to respect https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html .
It's really good to have profile support from official AWS CLI. I have many profiles set and want to reuse them
