incorrectly encoding + in urls
i am running into a prooblem where khttp double encodes data where the webserver returns already encoded data
i solved that temporarily by handling redirects myself and decoding the location header,
but it fails for '+' because khttp does not encode it properly, but double encodes it if i leave it encoded as ' %2B' , the result is then %25B2
(to be fair i wish there was no filenames contianing literal + buit nothing i can do there..)
not sure if this a fault at khttp or the webserver in question.. or both..
the following four urls are bebehaving strangely.. i can downlaod them fine using chrome.. but using khttp the redirects lead to invalid locations
sample:
var r = get(entry.url, allowRedirects = false)
while(r.statusCode == 302) {
val url = URLDecoder.decode(r.headers["Location"]!!, "UTF-8")
logger.info("following to {}", url)
r = get(url, allowRedirects = false)
}
if(r.statusCode == 200)
{
cacheFile.writeBytes(r.content)
} else {
logger.error("invalid statusCode {} from {}", r.statusCode, entry.url)
logger.error("connection url: {}",r.connection.url)
logger.error("content: {}", r.text)
return
}
comparison of some apparently wrong r.connection.url
thats what khttp does.. fails
get https://files.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245+69d0d70.jar
-> https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245%2B69d0d70.jar
-> https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245%252B69d0d70.jar
-> 403
and now lets try to guide it a little.. fail
get https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245+69d0d70.jar
-> https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245+69d0d70.jar
-> 403
this is what it should do
get https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245+69d0d70.jar
-> https://media.forgecdn.net/files/2443/194/BetterBuildersWands-1.12-0.11.1.245%2B69d0d70.jar
-> 200
it seems like the urlencoded % is getting encoded again, but replacing it with a + does not work either because that does not get properly urlencoded into %2B
it seems to happen both when following redirects automatically and manually
can confirm this with other URLS like from Thingiverse
https://cdn.thingiverse.com/zipfiles/4c/be/1d/e7/c3/Bottle+Opener+and+Cap+GUN%21+.zip
will then be
https://cdn.thingiverse.com/zipfiles/4c/be/1d/e7/c3/Bottle+Opener+and+Cap+GUN%2521+.zip
The Percentage sign will be encoded, even if it itself is for encoding another symbol