aws-sig4
aws-sig4 copied to clipboard
Incorrect url double encoding for path params
While attempting to sign a request to ElasticSearch in which a document id contained a "/" an incorrect signature was produced, resulting in the call to fail.
The failed case looked like so: https://search-my-es-domain.us-east-1.es.amazonaws.com/index/_doc/foo%2Fbar
The "foo%2Fbar" id is the correct url encoding of the document with internal ElasticSearch id of "foo/bar".
Per This document from Amazon the correct behavior is that "Each path segment must be URI-encoded twice."
This means the canonical-url should encode the above id as "foo%252Fbar" with a canonical uri of:
(canonical-uri "/index/_doc/foo%2Fbar")
=> "/index/_doc/foo%252Fbar"
I was able to get my request to succeed with the following code changes to aws-sig4.auth/canonical-uri
and I believe this is a more correct solution:
(defn double-encode [uri]
(cs/join "/" (map codec/url-encode (cs/split uri #"/"))))
(defn canonical-uri [uri]
(if (empty? uri)
"/"
(let [normalized (-> uri double-encode pathetic/normalize)]
(cond-> normalized
(and (pos? (count normalized)) (.endsWith uri "/"))
(str "/")))))
;Added to my own code to hotfix
(alter-var-root #'aws-sig4.auth/canonical-uri (constantly canonical-uri))
I attempted to create a PR but certain test fail, including the following path "/%20/foo". However, I am unsure if the test is correct.
@markbastian Sorry for late response. We've retired this library from internal use so it doesn't get actively monitored. That said, I'd be happy to take a PR and have a look at the test failure. Can you open?