openid
Charset of application/x-www-form-urlencoded requests/responses
Imported from AB/Connect bitbucket: https://bitbucket.org/openid/connect/issues/2055
Original Reporter: josephheenan
VP defines various http messages that have the content-type application/x-www-form-urlencoded.
I cannot see anything that specifies the charset. Normally I think this would not matter too much as the content is normally ASCII - however I think presentation_definition and presentation_submission can contain non-ASCII characters (couldn’t spot anything in https://identity.foundation/presentation-exchange/ that says anything either way) so we should explicitly state UTF-8 charset to be used I think.
Imported from AB/Connect bitbucket - Original Commenter: pwlb
The default webclient implementation in Kotlin sets the content-type “application/x-www-form-urlencoded;charset=UTF-8”, that’s how the issue came up in the conformance tests
@jogu, https://url.spec.whatwg.org/#urlencoded-parsing gives a very precise description (I've seen so far) with what we're dealing here
The application/x-www-form-urlencoded format is in many ways an aberrant monstrosity, the result of many years of implementation accidents and compromises leading to a set of requirements necessary for interoperability, but in no way representing good design practices. In particular, readers are cautioned to pay close attention to the twisted details involving repeated (and in some cases nested) conversions between character encodings and byte sequences. Unfortunately the format is in widespread use due to the prevalence of HTML forms.
and
A legacy server-oriented implementation might have to support encodings other than UTF-8 as well as have special logic for tuples of which the name is
_charset.
On one hand, it's a tiny detail hardly anyone pays attention too, but on the other hand, there's all the time wasted on troubleshooting rare but painful issues.
