NUL and CR incorrectly permitted within header values

[kenballus]

❓ I'm submitting a ...

• [X] 🐞 bug report
• [ ] 🐣 feature request
• [ ] ❓ question about the decisions made in the repository

🐞 Describe the bug. What is the current behavior? Cheroot allows CR (\r) and NUL (\x00) within header values.

❓ What is the motivation / use case for changing the behavior? RFC 9110 says not to do this:

Field values containing CR, LF, or NUL characters are invalid and dangerous, due to the varying ways that implementations might parse and interpret those characters; a recipient of CR, LF, or NUL within a field value MUST either reject the message or replace each of those characters with SP before further processing or forwarding of that message.

💡 To Reproduce Steps to reproduce the behavior:

1. Start a Cheroot-based HTTP server that echos received headers. (e.g., this one)
2. Send a request with a null byte and a carriage return in a header value (e.g., GET / HTTP/1.1\r\nHost: whatever\r\nTest: a\x00a\ra\r\n\r\n)
3. Observe that Cheroot processes the request, and the headers are echoed back with the NUL and CR intact.

💡 Expected behavior The request should be rejected, or the CR and NUL should be replaced with SP before Cheroot processes the request.

📋 Environment

• Cheroot version: 10.0.2.dev71+g1ff20b18
• Python version: 3.11.9
• OS: Linux 6.9.6

[webknjaz]

RFC 9110 says not to do this

FWIW, the newest RFCs were never implemented in here. So it might make sense to have a list of things differing from the older RFCs and have them addressed.