s3cmd icon indicating copy to clipboard operation
s3cmd copied to clipboard

s3cmd get --preserve does not appear to be implemented

Open jspv opened this issue 4 years ago • 1 comments

After struggling for a few hours trying to figure out why my s3cmd get --preserve commands were not setting the owner/group and permissions on my files which were uploaded with s3cmd put --preserve I looked through the code:

for s3cmd sync in cmd_sync_remote2local(), the following is implemented in the _download() which sets the mode, uid, gid, etc.

           if 's3cmd-attrs' in response and cfg.preserve_attrs:
                    attrs = response['s3cmd-attrs']
                    if 'mode' in attrs:
                        os.chmod(deunicodise(dst_file), int(attrs['mode']))
                    if 'mtime' in attrs or 'atime' in attrs:
                        mtime = ('mtime' in attrs) and int(attrs['mtime']) or int(time.time())
                        atime = ('atime' in attrs) and int(attrs['atime']) or int(time.time())
                        os.utime(deunicodise(dst_file), (atime, mtime))
                    if 'uid' in attrs and 'gid' in attrs:
                        uid = int(attrs['uid'])
                        gid = int(attrs['gid'])
                        os.lchown(deunicodise(dst_file),uid,gid)
                elif 'last-modified' in response['headers']:
                    last_modified = time.mktime(time.strptime(response["headers"]["last-modified"], "%a, %d %b %Y %H:%M:%S GMT"))
                    os.utime(deunicodise(dst_file), (last_modified, last_modified))
                    debug("set mtime to %s" % last_modified)

in cmd_object_get() the following sets the modified times, but there appears to be no code to deal with the mode, uid, git, etc., but no code that parses 's3cmd-attrs' and sets them according to what is expected by --preserve

        if "x-amz-meta-s3tools-gpgenc" in response["headers"]:
            gpg_decrypt(destination, response["headers"]["x-amz-meta-s3tools-gpgenc"])
            response["size"] = os.stat(deunicodise(destination))[6]
        if "last-modified" in response["headers"] and destination != "-":
            last_modified = time.mktime(time.strptime(response["headers"]["last-modified"], "%a, %d %b %Y %H:%M:%S GMT"))
            os.utime(deunicodise(destination), (last_modified, last_modified))
            debug("set mtime to %s" % last_modified)

Note: the appropriate code appears to be in s3cmd put to set the headers appropriately.

        attr_header = _build_attr_header(local_list, key)
        debug(u"attr_header: %s" % attr_header)
        extra_headers.update(attr_header)

jspv avatar May 25 '20 14:05 jspv

I have the same issue with version 2.1.0 (with the Scaleway storage service)

ALRBP avatar Jan 25 '21 18:01 ALRBP