"aws s3 cp" download to /dev/null gives back a warning, a success and an error

[atsalolikhin-spokeo]

I've opened an AWS support case (Case ID 7148947761) and the AWS support rep said the service team asked me to open a GitHub issue about this.

I am working with S3 kms-key-policy, and in order to test my changes, I went to download a file using awscli (aws s3 cp).

I didn't actually want the file, so I put "/dev/null" as the local path (like "curl -O /dev/null http://example.com/my-file.txt ").

This tickled some bug in aws-cli, because it started throwing errors:

$ aws s3 cp s3://[redacted] /dev/null
warning: Skipping file /dev/null. Successfully Downloaded /dev/null but was unable to update the last modified time. [Errno 13] Permission denied
download: s3://[redacted] to ../../../../../../../../dev/null
$

There is a warning, a success, and an error. It looks like it actually transferred the file however.

This is a bit confusing and raises some concern about the integrity of the innards of aws-cli.

There is no production impact.

This is with aws-cli/2.0.19

The support engineer wrote:

When downloading with the aws s3 sync/cp commands the AWS CLI will try to update the file(s) modified time to match the S3 object key modified time. When not root the OS returns permission-denied/permission-problem when AWS CLI tries to do this for the /dev/null file which is why warning (OS error) is being returned. They could also share an AWS github code describing how it works here: https://github.com/aws/aws-cli/blob/73f321d1e72d25ee4200eac59954b4e2d5a9c573/awscli/customizations/s3/utils.py#L702

[KaibaLopez]

Hmm, yea this is a kind of an edge case and you're right in that the output is confusing. I'll mark it as a bug for now and look more into it. Thanks for bringing it up to us.

[atsalolikhin-spokeo]

Sure! You are welcome! :)

[m-31]

This is also a use case for us. We need to check if we have the permission copy a certain path (aws s3 ls and other calls are not permitted). The only way we found so far is calling aws s3 cp bucket/path .

[lazize]

Any update about this?

[jason-harrison-weir]

I encountered this issue performing an aws s3 cp to a ubuntu box with a cifs mounted azure storage volume.

Mount configuration is: rw,relatime,vers=3.0,cache=strict,upcall_target=app,username=[REDACTED],uid=0,noforceuid,gid=0,noforcegid,addr=[REDACTED],file_mode=0766,dir_mode=0777,soft,persistenthandles,nounix,serverino,mapposix,reparse=nfs,nativesocket,symlink=native,rsize=1048576,wsize=1048576,bsize=1048576,retrans=1,echo_interval=60,actimeo=1,closetimeo=1

Our workaround is to copy a single file at a time.