Fix decompression in web cmdlets

[iSazonov]

PR Summary

Follow doesn't work for original cmdlet:

Invoke-WebRequest https://httpbin.org/deflate

I don't know why our tests are passed.

I guess there was something changed .Net from early versions.

The fix removes old workaround and turn on embedded automatic decompression in HttpClient handler.

PR Context

PR Checklist

• [x] PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• [x] Summarized changes
• [x] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
• [x] This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
• Breaking changes
  • [x] None
  • OR
  • [ ] Experimental feature(s) needed
    • [ ] Experimental feature name(s):
• User-facing changes
  • [x] Not Applicable
  • OR
  • [ ] Documentation needed
    • [ ] Issue filed:
• Testing - New and feature
  • [x] N/A or can only be tested interactively
  • OR
  • [ ] Make sure you've added a new test if existing tests do not effectively test the code changed
• Tooling
  • [ ] I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
  • OR
  • [ ] I have considered the user experience from a tooling perspective and opened an issue in the relevant tool repository. This may include:
    • [ ] Impact on PowerShell Editor Services which is used in the PowerShell extension for VSCode (which runs in a different PS Host).
      • [ ] Issue filed:
    • [ ] Impact on Completions (both in the console and in editors) - one of PowerShell's most powerful features.
      • [ ] Issue filed:
    • [ ] Impact on PSScriptAnalyzer (which provides linting & formatting in the editor extensions).
      • [ ] Issue filed:
    • [ ] Impact on EditorSyntax (which provides syntax highlighting with in VSCode, GitHub, and many other editors).
      • [ ] Issue filed:

[PaulHigin]

@iSazonov

Overall, these changes look Ok to me. But I don't know what prompted this PR. Did the tests begin failing at some point? Or is this just code clean up to remove the current workaround and instead use the built-in automatic decompression setting?

[iSazonov]

@PaulHigin The Invoke-WebRequest https://httpbin.org/deflate did not work before the PR but our tests for deflate was passed. And yes, .Net Runtime remove encoding headers after automatic decompression. I was wonder but it makes sense - if there was follow processing it shouldn't see any compression header since content is already not compressed. I checked this with Wireshark:

1. Request

2. Response

Then the cmdlets return the response without Content-Encoding: deflate.

---

Second issue was with tests. I tried to keep them as near to original as possible. But I started to check with httpbin.org. The site send another response than our WebListener. httpbin.org response has a flag for used decompression method, WebListener haven't. I don't know how enhance WebListener. So I had to change tests.

---

I don't know why our tests passed before the PR for deflate. I did not investigate either a root is in WebListener or in old workaround.

[ghost]

This pull request has been automatically marked as Review Needed because it has been there has not been any activity for 7 days. Maintainer, please provide feedback and/or mark it as Waiting on Author

[ghost]

This pull request has been automatically marked as Review Needed because it has been there has not been any activity for 7 days. Maintainer, please provide feedback and/or mark it as Waiting on Author

[pull-request-quantifier-deprecated[bot]]

This PR has 22 quantified lines of changes. In general, a change size of upto 200 lines is ideal for the best PR experience!

---

Quantification details

Label      : Extra Small
Size       : +8 -14
Percentile : 8.8%

Total files changed: 3

Change summary by file extension:
.cs : +1 -7
.ps1 : +7 -7

Change counts above are quantified counts, based on the PullRequestQuantifier customizations.

Why proper sizing of changes matters

Optimal pull request sizes drive a better predictable PR flow as they strike a balance between between PR complexity and PR review overhead. PRs within the optimal size (typical small, or medium sized PRs) mean:

• Fast and predictable releases to production:
  • Optimal size changes are more likely to be reviewed faster with fewer iterations.
  • Similarity in low PR complexity drives similar review times.
• Review quality is likely higher as complexity is lower:
  • Bugs are more likely to be detected.
  • Code inconsistencies are more likely to be detected.
• Knowledge sharing is improved within the participants:
  • Small portions can be assimilated better.
• Better engineering practices are exercised:
  • Solving big problems by dividing them in well contained, smaller problems.
  • Exercising separation of concerns within the code changes.

What can I do to optimize my changes

• Use the PullRequestQuantifier to quantify your PR accurately
  • Create a context profile for your repo using the context generator
  • Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out the Excluded section from your prquantifier.yaml context profile.
  • Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in your prquantifier.yaml context profile.
  • Only use the labels that matter to you, see context specification to customize your prquantifier.yaml context profile.
• Change your engineering behaviors
  • For PRs that fall outside of the desired spectrum, review the details and check if:
    • Your PR could be split in smaller, self-contained PRs instead
    • Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).

How to interpret the change counts in git diff output

• One line was added: +1 -0
• One line was deleted: +0 -1
• One line was modified: +1 -1 (git diff doesn't know about modified, it will interpret that line like one addition plus one deletion)
• Change percentiles: Change characteristics (addition, deletion, modification) of this PR in relation to all other PRs within the repository.

---

Was this comment helpful? :thumbsup:  :ok_hand:  :thumbsdown: (Email) Customize PullRequestQuantifier for this repository.

[daxian-dbw]

@PaulHigin Can you please take another look?

[CarloToso]

This PR also fixes Invoke-WebRequest https://httpbin.org/brotli

[God-damnit-all]

Is this going to make it into 7.4?

[iSazonov]

It is already in v7.4.0-preview.1

[God-damnit-all]

It is already in v7.4.0-preview.1

Oh, cool. Normally a bot responds to the PR saying if it made it in or whatever.