sorted headers

[fayland]

• Mojolicious version: ALL
• Perl version: ALL
• Operating system: ALL

Steps to reproduce the behavior

if you do with curl or Chrome or whatever browser, the headers are always sorted as typed. but in Perl since headers are hash, the order is somehow not desired

Expected behavior

header Host is before header Accept for example.

Actual behavior

Host might be the last in Mojo::Headers::to_string

we can try to add "sorted_headers" as ArrayRef and fix the return part for 'sub names' in Mojo::Headers

if you need a patch, I can submit one. but first of all, please agree that's acceptable.

Thanks

[kraih]

What is the problem you are trying to solve?

[karenetheridge]

references:

• https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html "The order in which header fields with differing field names are received is not significant. However, it is "good practice" to send general-header fields first, followed by request-header or response- header fields, and ending with the entity-header fields. "

• HTTP::Headers follows the "good practice" order: https://metacpan.org/pod/HTTP::Headers#$h-%3Eheader_field_names https://metacpan.org/dist/HTTP-Message/source/lib/HTTP/Headers.pm#L14-63

[kraih]

@karenetheridge Ordering headers that way is very expensive, we are not going to do that.

[karenetheridge]

This could be enabled optionally (defaulting to off), so those who want the ordered headers at the expense of performance can have it.

[fayland]

one of my issue is that CloudFlare actually detects the order of headers. and return 403 forbidden if the order is not correct. (I can provider example if you need it)

can make it optional like has 'ordered_headers'; or pass { Host => 'a', Referer => 'b', __sorted => ['Host', 'Referer'] } as headers.

Thanks

[s1037989]

Could this be done with a hook and therefore a plugin?

[jhthorsen]

@fayland: In which case does it return a 403? I've been running my Mojo apps behind Cloudflare for many years and never experienced what you are saying.

[fayland]

CODE REMOVED

Thanks

[jhthorsen]

@fayland: What I meant is that there's probably a some firewall rule (or another setting) in Cloudflare that has been enabled for this to happen, no..?

[fayland]

not sure. I have to overwrite Mojo::Headers::to_string to make it working right now. thinking it might be good if we can do a feature to make header sortable. if it's not acceptable, it's alright as well since I can do it with *Mojo::Headers::to_string

Thanks

[jhthorsen]

It does sound a bit random though. Maybe just apply a role for this case?

# Make a custom to_string()
package Mojo::Headers::Role::Sorted;
use Mojo::Base -role, -signatures;
around to_string => sub ($next, $self) { ... };

# And then apply it to the user agent:
package main;
my $ua = Mojo::UserAgent->new;
$ua->on(start => sub ($ua, $tx) { $tx->req->headers->with_roles('+Sorted') });

[kraih]

I can replicate the problem with the test case, it fails randomly based on header order. What i find interesting is that even just alphabetical order of headers makes it pass.

[karenetheridge]

does anyone know anyone at CloudFlare? This is such a flagrant violation of the RFCs, and it would actually take a lot of effort to fail like this; what on earth could the justification be for it??

[kraih]

Could be WebSocket specific, given the test case. Handshakes are easy to get wrong given their relationship to HTTP 1.1. But anyway, i'm happy to try alphabetical headers to make our generated HTTP messages more canonical. https://github.com/mojolicious/mojo/commit/64354fac02f63a6cb023076080eaf5d6432d6562

[karenetheridge]

Alphabetical is not the "good practice" order though. https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html

[kraih]

Alphabetical is not the "good practice" order though. https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html

It's canonical and fast though.