Multipart Invalid boundary in C-T (characters) error thrown when charset follows boundary element in CT header

[bitsofinfo]

If a this Content Type header is sent the error Multipart: Invalid boundary in C-T (characters) is thrown Content-Type: multipart/form-data; boundary=09WYPQJGmFFxPW2w0gbsmeeo3AX45nRC-; charset=UTF-8

If this Content Type header is sent: the error Multipart: Invalid boundary in C-T (characters) is NOT thrown Content-Type: multipart/form-data; charset=UTF-8; boundary=09WYPQJGmFFxPW2w0gbsmeeo3AX45nRC-

https://github.com/SpiderLabs/ModSecurity/blob/f66cd4111f08a26d4429cb0071e992deb050928a/apache2/msc_multipart.c#L893

[victorhora]

Hi @bitsofinfo,

Not sure which version of ModSecurity (v2?) you are using, but both cases results in the same error for me (including on libModSecurity)

REQUEST 1:

--02e9ee56-B--
POST / HTTP/1.1
HOST: host.example.com
Cookie: some_cookies...
Connection: Keep-Alive

Content-Type: multipart/form-data; charset=UTF-8;

boundary=09WYPQJGmFFxPW2w0gbsmeeo3AX45nRC-
Content-Length: 147

--02e9ee56-C--
--09WYPQJGmFFxPW2w0gbsmeeo3AX45nRC-
Content-Disposition: form-data; name="boom"

this is going to explode
--09WYPQJGmFFxPW2w0gbsmeeo3AX45nRC---
--02e9ee56-I--

--02e9ee56-H--

Message: Multipart parsing error (init): Multipart: Invalid boundary in C-T (malformed). Message: Access denied with code 400 (phase 2). Match of "eq 0" against "REQBODY_ERROR" required. [file "/etc/modsecurity/modsecurity.conf"] [line "67"] [id "200002"] [msg "Failed to parse request body."] [data "Multipart: Invalid boundary in C-T (malformed)."] [severity "CRITICAL"]

Apache-Error: [file "apache2_util.c"] [line 271] [level 3] [client 192.168.37.1] ModSecurity: Multipart parsing error (init): Multipart: Invalid boundary in C-T (malformed). [hostname "host.example.com"] [uri "/"] [unique_id "W33eI38AQEAABlYj8EAAABN"]
Apache-Error: [file "apache2_util.c"] [line 271] [level 3] [client 192.168.37.1] ModSecurity: Access denied with code 400 (phase 2). Match of "eq 0" against "REQBODY_ERROR" required. [file "/etc/modsecurity/modsecurity.conf"] [line "67
"] [id "200002"] [msg "Failed to parse request body."] [data "Multipart: Invalid boundary in C-T (malformed)."] [severity "CRITICAL"] [tag "DROP on phase 4!"] [hostname "host.example.com"] [uri "/"] [unique_id "W33eI38AAQEAABlYj8EAAABN
"]
Action: Intercepted (phase 2)
Stopwatch: 1534975523934976 575 (- - -)
Stopwatch2: 1534975523934976 575; combined=97, p1=60, p2=24, p3=0, p4=0, p5=13, sr=0, sw=0, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.9.2 (http://www.modsecurity.org/).
Server: Apache/2.4.10 (Debian)
Engine-Mode: "ENABLED"

REQUEST 2:

--a5003968-B--
POST / HTTP/1.1
HOST: host.example.com
Cookie: some_cookies...
Connection: Keep-Alive

Content-Type: multipart/form-data; boundary=09WYPQJGmFFxPW2w0gbsmeeo3AX45nRC-; charset=UTF-8

Content-Length: 147

--a5003968-C--
--09WYPQJGmFFxPW2w0gbsmeeo3AX45nRC-
Content-Disposition: form-data; name="boom"

this is going to explode
--09WYPQJGmFFxPW2w0gbsmeeo3AX45nRC---
--a5003968-I--

--a5003968-H--

Message: Multipart parsing error (init): Multipart: Invalid boundary in C-T (characters).

Message: Access denied with code 400 (phase 2). Match of "eq 0" against "REQBODY_ERROR" required. [file "/etc/modsecurity/modsecurity.conf"] [line "67"] [id "200002"] [msg "Failed to parse request body."] [data "Multipart: Invalid boundary in C-T (characters)."] [severity "CRITICAL"] [tag "DROP on phase 4!"]
Apache-Error: [file "apache2_util.c"] [line 271] [level 3] [client 192.168.37.1] ModSecurity: Multipart parsing error (init): Multipart: Invalid boundary in C-T (characters). [hostname "host.example.com"] [uri "/"] [unique_id "W33gcH8A
AQEAABlYj8IAAABO"]
Apache-Error: [file "apache2_util.c"] [line 271] [level 3] [client 192.168.37.1] ModSecurity: Access denied with code 400 (phase 2). Match of "eq 0" against "REQBODY_ERROR" required. [file "/etc/modsecurity/modsecurity.conf"] [line "67
"] [id "200002"] [msg "Failed to parse request body."] [data "Multipart: Invalid boundary in C-T (characters)."] [severity "CRITICAL"] [tag "DROP on phase 4!"] [hostname "host.example.com"] [uri "/"] [unique_id "W33gcH8AAQEAABlYj8IAAAB
O"]
Action: Intercepted (phase 2)
Stopwatch: 1534976112788216 734 (- - -)
Stopwatch2: 1534976112788216 734; combined=126, p1=84, p2=34, p3=0, p4=0, p5=8, sr=0, sw=0, l=0, gc=0
Response-Body-Transformed: Dechunked
Producer: ModSecurity for Apache/2.9.2 (http://www.modsecurity.org/).
Server: Apache/2.4.10 (Debian)
Engine-Mode: "ENABLED"

---

Can you elaborate more? Also if you have an improvement in mind, you're more than welcome to submit a pull request :)

[bitsofinfo]

2.9.2 only happens as described for me. I have an external client library generating these requests. (an apache commons lib doing the POSTs)

Not much more to elaborate, but the existence of a charset in the CT line regardless of if it precedes or follows the boundary should not cause this happen; no?

I would but unfortunately I'm not C programmer :)

[beatsandpics]

Any progress on this and a potential 2.9.2 fix? (not using 3.x)

[osamamaruf]

@victorhora This happens to me regardless of the position of the charset=UTF-8; placement in the Content-Type header as you have observed as well. We are using spring and spring add this header to all multi part requests. Only removing encoding format works. We can remove the encoding or add an exclusion but should not this be allowed by default?

[martinhsv]

Hi @osamamaruf ,

Agreed, the issue described here is legitimate. The charset= should not be flagged as an error.

We haven't forgotten about the issue. It's currently on the to-do list for 3.1.1.

[mschulze0]

Has this ever been fixed? Is there a workaround?

[tandarim]

bump up, we stumbled today over this issue, so same question: was this fixed in 3.0.4?

[martinhsv]

This has not been addressed yet. It had been scheduled to be done as part of v3.1.x. I'll see if I can up the priority on this.

[martinhsv]

A bit of an update on this item ...

Having a charset= parameter in that Content-Type header appears to be contrary to the standard.

See the reference Optional parameters: none in https://www.iana.org/assignments/media-types/multipart/form-data .

And here is the equivalent passage in the relevant RFC: https://datatracker.ietf.org/doc/html/rfc7578#section-8

The above, in combination with the definition of the Content-Type header ( https://datatracker.ietf.org/doc/html/rfc1521#section-4 ), would seem to lead to that conclusion.

Even if we are agreed that having the charset= parameter there is non-conformant, that doesn't mean we cannot go ahead and adjust ModSecurity to be permissive with such requests. We have done such things before when some non-malicious client-side frameworks are known to send the non-conformant bytes. But it might mean proceeding somewhat more warily.

[martinhsv]

For v3, a pull request available for this at https://github.com/SpiderLabs/ModSecurity/pull/2731 has been merged.

If anyone using ModSecurity v3 wants to try it out before the next release is official, this could be a good time to do so.

[martinhsv]

This was implemented via #2731 .