azure-sdk-for-python icon indicating copy to clipboard operation
azure-sdk-for-python copied to clipboard

[Storage] Change algorithm used to sort headers

Open vincenttran-msft opened this issue 1 year ago • 3 comments

This PR aims to solve the differences between how Python's built-in sort algorithm sorts the headers compared to the service. Although this is usually not problematic as headers are quite contrived and restricted to what values they can be, the issue arises with metadata where it allows user input to be part of the headers we are going to sort. #10277

With that in mind, we discovered that only ASCII values are acceptable on the service side, making our acceptable character-space much smaller.

Then we can eliminate ASCII values 0-32 as they are nonsensical (NULL, LF, Space, etc.) and also 127 (DEL).

This leaves us with the following characters (following ASCII value sorting scheme): "!"#$%&'()*+,./-^_`0123456789:;<=>? @ ABCDEFGHIJKLMNOPQRSTUVWXYZ[]abcdefghijklmnopqrstuvwxyz{|}~"

Now, we need to check what values are accepted once we send the headers off for authentication. And so, the following characters either fail in the _is_legal_header_name check or simply throws a Bad Request (comma separated): :, ", `, ^, (, ), comma, /, , ;, <, =, >, ?, @, [, ], {, }

Now at this point, if we use our pared-down list of the ASCII values and remove all the illegal characters as stated above, the sort will still fail if we use these weights (ASCII-value as the weight), as the service does not follow ASCII value weight convention.

And so, after much testing, we finally land on this as the proper weighting to match how service is sorting the characters: "-!#$%&*.^_|~+"'(),/`~0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]abcdefghijklmnopqrstuvwxyz{}"

vincenttran-msft avatar Oct 21 '22 23:10 vincenttran-msft

API change check

API changes are not detected in this pull request.

azure-sdk avatar Oct 22 '22 00:10 azure-sdk

/azp run python - storage - tests

vincenttran-msft avatar Nov 07 '22 22:11 vincenttran-msft

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Nov 07 '22 22:11 azure-pipelines[bot]

/check-enforcer override

vincenttran-msft avatar Nov 17 '22 00:11 vincenttran-msft