moto icon indicating copy to clipboard operation
moto copied to clipboard

Bucket.download_file downloads file with metadata

Open andrescevp opened this issue 3 years ago • 0 comments

I am uploading a file and then downloading the same file.

The upload works fine, I am checking the content and is there where it fails

The file:

ID|COUNTRY
1|Foo
...

The upload code

s3_client.upload_file(file_name, bucket, object_name, ExtraArgs={"Metadata": {"foo": "boo"}, "ChecksumAlgorithm": "SHA256"})

The test failing

body = conn.Object(S3_BUCKET, "my_key.csv").get()['Body'].read().decode("utf-8")
assert body.startswith('ID|COUNTRY'), body
>>>
assert False
E        +  where False = <built-in method startswith of str object at 0x55a06e2966e0>('ID|COUNTRY')
E        +    where <built-in method startswith of str object at 0x55a06e2966e0> = '2f3\r\nID|COUNTRY...'

I would expect to have exactly the same body as I am saving via pandas.to_csv

The 2f3 is a random value as far I could see...

The downloaded files look like

E       AssertionError: 2f3 # this is new content
E         ID|COUNTRY
E         1|US
E         ...
E         # this is new content
E         0 # this is new content
E         x-amz-checksum-sha256:0QMsyrLjuMEaWcVrl26PZwuf0axTOR2XdUcU1uD0ayQ= # this is new content - from the meta? and where is my extra meta?

I am working around with...

# only in test suite
with open(_tmp_file, 'rb') as tf:
    content = tf.readlines()[1:(len(tf.readlines())-3)]
with open(_tmp_file, 'wb') as tfw:
    tfw.write(b'')
    tfw.writelines(content)

andrescevp avatar Sep 23 '22 15:09 andrescevp