aws-sig4 icon indicating copy to clipboard operation
aws-sig4 copied to clipboard

Incorrect url double encoding for path params

Open markbastian opened this issue 4 years ago • 1 comments

While attempting to sign a request to ElasticSearch in which a document id contained a "/" an incorrect signature was produced, resulting in the call to fail.

The failed case looked like so: https://search-my-es-domain.us-east-1.es.amazonaws.com/index/_doc/foo%2Fbar

The "foo%2Fbar" id is the correct url encoding of the document with internal ElasticSearch id of "foo/bar".

Per This document from Amazon the correct behavior is that "Each path segment must be URI-encoded twice."

This means the canonical-url should encode the above id as "foo%252Fbar" with a canonical uri of:

(canonical-uri "/index/_doc/foo%2Fbar")
=> "/index/_doc/foo%252Fbar"

I was able to get my request to succeed with the following code changes to aws-sig4.auth/canonical-uri and I believe this is a more correct solution:

(defn double-encode [uri]
  (cs/join "/" (map codec/url-encode (cs/split uri #"/"))))

(defn canonical-uri [uri]
  (if (empty? uri)
    "/"
    (let [normalized (-> uri double-encode pathetic/normalize)]
      (cond-> normalized
              (and (pos? (count normalized)) (.endsWith uri "/"))
              (str "/")))))

;Added to my own code to hotfix
(alter-var-root #'aws-sig4.auth/canonical-uri (constantly canonical-uri))

I attempted to create a PR but certain test fail, including the following path "/%20/foo". However, I am unsure if the test is correct.

markbastian avatar Mar 30 '20 18:03 markbastian

@markbastian Sorry for late response. We've retired this library from internal use so it doesn't get actively monitored. That said, I'd be happy to take a PR and have a look at the test failure. Can you open?

ovan avatar Apr 21 '20 13:04 ovan