clyso
[BUG] chorus proxy does not handle slashes and other critical keys correctly
Bug Report
Description
The chorus proxy does not handle slashes and other object keys which look like directories or paths correctly.
The s3 specification is quite vague about this topic and states that slashes and dots are valid values in object keys which might need special handling. Nevertheless, at least AWS, the commercial cloudian s3 solution and the open-source ceph radosdw implementation support arbitrary dots and slashes in keys.
For example '.', './', '././', '/', '///', '/foo', 'foo/', '//foo', 'foo//', '/foo/' and '//foo//' are all totally valid, distinct object identifiers.
Expected Behavior
The chorus proxy can handle the mentioned critical keys.
Actual Behavior
Depending on the provided key, chorus either redirects the client to a location with a sanitized path or fails to authenticate the request because of trimmed slashes, which lead to a signature mismatch.
Steps to Reproduce
PUT an objects with the keys like '/', '//', or '././././'
Environment
- Chorus Version: commit 71188d5
See https://github.com/doomguy-stg/chorus/tree/critical_object_keys for tests and a possible solution.
@doomguy-stg thank you for detailed report. can you please give an example of invalid object key so we can reproduce it in e2e test?
am i correct, that these one are failing on chorus proxy and working correctly directly on s3?
- PUT /mybucket//
-
PUT /mybucket//. -
PUT /mybucket/. -
PUT /mybucket/./ - basically, anything ending with
/
It is actually anything with trailing or leading slashes (due to the strings.Trim() in req_parser.go) plus anything affected by URL preprocessing provided by http.ServerMux (mainly https://pkg.go.dev/net/http#hdr-Trailing_slash_redirection-ServeMux and https://pkg.go.dev/net/http#hdr-Request_sanitizing-ServeMux).
See Commit 6184684 for tests (not all of them fail, but a lot) and Commit aa07210 for a possible solution.
Unfortunately, rclone does some key-manipulation, too. Some objects are not synced properly. The situation has improved with the switch to rclone 1.70.3, but (at least) keys containing single or double dots are still affected. (See the test updates in commit https://github.com/clyso/chorus/commit/aa07210bf5f34c5362434b7619502eca45b29b7e)
The rclone behavior is, in my opinion, totally fine from rclones point of view. It might be possible to fix this in rclone, but it might also be more productive to get rid of rclone and choose or implement an independent solution. But that is another story...