s3proxy
s3proxy copied to clipboard
Published
20 hours ago •
gaul
gaul
S3Proxy returns Internal Server Error when using invalid Azure metadata keys
When issuing a PUT request with an invalid Azure metadata key (one that contains - in this example), S3Proxy returns an Internal Server error (as opposed to the 401 Unauthorized that Azure returns).
Here is the output from s3cmd with debug turned on:
s3cmd --no-ssl --debug put dumper.py s3://test/
DEBUG: String 'b'root'' encoded to 'root'
DEBUG: String 'b'root'' encoded to 'root'
DEBUG: attr_header: {'x-amz-meta-s3cmd-attrs': 'atime:1506556690/ctime:1506466030/gid:0/gname:root/md5:39c14bd9c4278fde49d793f180b99a9f/mode:33261/mtime:1506465897/uid:0/uname:root'}
DEBUG: DeUnicodising 'dumper.py' using UTF-8
DEBUG: DeUnicodising 'dumper.py' using UTF-8
DEBUG: DeUnicodising 'dumper.py' using UTF-8
DEBUG: CreateRequest: resource[uri]=/dumper.py
upload: 'dumper.py' -> 's3://test/dumper.py' [1 of 1]
DEBUG: DeUnicodising 'dumper.py' using UTF-8
DEBUG: Using signature v2
DEBUG: SignHeaders: 'PUT\n\ntext/plain\n\nx-amz-date:Fri, 06 Oct 2017 19:06:52 +0000\nx-amz-meta-s3cmd-attrs:atime:1506556690/ctime:1506466030/gid:0/gname:root/md5:39c14bd9c4278fde49d793f180b99a9f/mode:33261/mtime:1506465897/uid:0/uname:root\nx-amz-storage-class:STANDARD\n/test/dumper.py'
DEBUG: get_hostname(test): localhost:20080
DEBUG: ConnMan.get(): creating new connection: http://localhost:20080
DEBUG: non-proxied HTTPConnection(localhost, 20080)
DEBUG: format_uri(): /test/dumper.py
434 of 434 100% in 0s 24.26 kB/sDEBUG: ConnMan.put(): connection put back to pool (http://localhost:20080#1)
DEBUG: Response:
{'data': b'<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html'
b';charset=ISO-8859-1"/>\n<title>Error 500 </title>\n</head>\n<body>\n'
b'<h2>HTTP ERROR: 500</h2>\n<p>Problem accessing /test/dumper.py. R'
b'eason:\n<pre> Server Error</pre></p>\n<hr /><i><small>Powered b'
b'y Jetty://</small></i>\n</body>\n</html>\n',
'headers': {'cache-control': 'must-revalidate,no-cache,no-store',
'content-length': '295',
'content-type': 'text/html; charset=ISO-8859-1',
'date': 'Fri, 06 Oct 2017 19:06:52 GMT',
'server': 'Jetty(9.2.z-SNAPSHOT)'},
'reason': 'Server Error',
'size': 434,
'status': 500}
434 of 434 100% in 0s 1542.80 B/s done
DEBUG: S3Error: 500 (Server Error)
DEBUG: HttpHeader: content-length: 295
DEBUG: HttpHeader: server: Jetty(9.2.z-SNAPSHOT)
DEBUG: HttpHeader: etag:
DEBUG: HttpHeader: cache-control: must-revalidate,no-cache,no-store
DEBUG: HttpHeader: date: Fri, 06 Oct 2017 19:06:52 GMT
DEBUG: HttpHeader: content-type: text/html; charset=ISO-8859-1
ERROR: Error parsing xml: Malformed error XML returned from remote server.. ErrorXML: b'<html>\n<head>\n<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>\n<title>Error 500 </title>\n</head>\n<body>\n<h2>HTTP ERROR: 500</h2>\n<p>Problem accessing /test/dumper.py. Reason:\n<pre> Server Error</pre></p>\n<hr /><i><small>Powered by Jetty://</small></i>\n</body>\n</html>\n'
WARNING: Upload failed: /dumper.py (500 (Server Error)
S3Proxy could also have an opt-in feature to work around this Azure limitation by encoding the characters in an accepted format (e.g. Base32 encoding, replacing = padding with _). However, at the very least it would be great to not get back an Internal Server Error.
