khttp icon indicating copy to clipboard operation
khttp copied to clipboard

incorrectly encoding + in urls

Open NikkyAI opened this issue 7 years ago • 1 comments

i am running into a prooblem where khttp double encodes data where the webserver returns already encoded data

i solved that temporarily by handling redirects myself and decoding the location header, but it fails for '+' because khttp does not encode it properly, but double encodes it if i leave it encoded as ' %2B' , the result is then %25B2

(to be fair i wish there was no filenames contianing literal + buit nothing i can do there..)

not sure if this a fault at khttp or the webserver in question.. or both..

the following four urls are bebehaving strangely.. i can downlaod them fine using chrome.. but using khttp the redirects lead to invalid locations

sample:

    var r = get(entry.url, allowRedirects = false)
    while(r.statusCode == 302) {
        val url = URLDecoder.decode(r.headers["Location"]!!, "UTF-8")
        logger.info("following to {}", url)
        r = get(url, allowRedirects = false)
    }
    if(r.statusCode == 200)
    {
       cacheFile.writeBytes(r.content)
    } else {
         logger.error("invalid statusCode {} from {}", r.statusCode, entry.url)
         logger.error("connection url: {}",r.connection.url)
         logger.error("content: {}", r.text)
         return
}

comparison of some apparently wrong r.connection.url

thats what khttp does.. fails

get https://files.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245+69d0d70.jar
 -> https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245%2B69d0d70.jar
 -> https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245%252B69d0d70.jar
-> 403

and now lets try to guide it a little.. fail

get https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245+69d0d70.jar
-> https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245+69d0d70.jar
-> 403

this is what it should do

get https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245+69d0d70.jar
-> https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245%2B69d0d70.jar
-> 200

it seems like the urlencoded % is getting encoded again, but replacing it with a + does not work either because that does not get properly urlencoded into %2B

it seems to happen both when following redirects automatically and manually

NikkyAI avatar Feb 08 '18 02:02 NikkyAI

can confirm this with other URLS like from Thingiverse https://cdn.thingiverse.com/zipfiles/4c/be/1d/e7/c3/Bottle+Opener+and+Cap+GUN%21+.zip

will then be https://cdn.thingiverse.com/zipfiles/4c/be/1d/e7/c3/Bottle+Opener+and+Cap+GUN%2521+.zip

The Percentage sign will be encoded, even if it itself is for encoding another symbol

Serverfrog avatar Aug 31 '20 16:08 Serverfrog