Invoke-WebRequest: Data Encoding Should Be Elaborated On

[SetTrend]

trafficstars

Prerequisites

• [X] Existing Issue: Search the existing issues for this repository. If there is an issue that fits your needs do not file a new one. Subscribe, react, or comment on that issue instead.
• [X] Descriptive Title: Write the title for this issue as a short synopsis. If possible, provide context. For example, "Typo in Get-Foo cmdlet" instead of "Typo."
• [X] Verify Version: If there is a mismatch between documentation and the behavior on your system, ensure that the version you are using is the same as the documentation. Check this box if they match or the issue you are reporting is not version specific.

Links

https://learn.microsoft.com/powershell/module/microsoft.powershell.utility/invoke-webrequest#parameters

Summary

The documentation lacks explanation of how parameter data is getting escaped.

Details

Here's an example pertaining the Body parameter:

How is data for the Body parameter to be provided? Should the form parameter string be URL-encoded? Or does the Invoke-WebRequest Cmdlet do this?

So, what is correct:

this?

Invoke-WebRequest -Body '[email protected]'

… or this?

Invoke-WebRequest -Body 'email=a%40b.com'

Suggested Fix

Please provide the missing information.

[michaeltlombardi]

Thanks for filing this, @SetTrend! Looking through the documentation, there's some stuff to clean up and clarify in the docs, but the short answer to your question around whether Invoke-WebRequest modifies text in Body is that Invoke-WebRequest doesn't modify the text values.

Some endpoints will accept non-URL-encoded data and some require it. If your endpoint requires the data values to be URL-encoded, you need to ensure you encode that information yourself. For example:

$EncodedEmail = [System.Net.WebUtility]::UrlEncode("[email protected]")
Invoke-WebRequest -Body "email=$EncodedEmail"

[SetTrend]

Thank you for clarifying, @michaeltlombardi !

How about this case:

When the input is a GET request and the body is an IDictionary (typically, a hash table), the body is added to the URI as query parameters.

I guess, this will auto UrlEncode?

[michaeltlombardi]

@SetTrend yup, because it's not posting a body but instead composing a query URI.

Invoke-WebRequest -Uri https://httpbin.org/get -Method Get -Body @{
  Foo = 'bar baz'
  Bar = 5
  Baz = [datetime]::Now
} | Select-Object -ExpandProperty Content

{                                                                                                                       
  "args": {
    "Bar": "5",
    "Baz": "10/19/2022 3:23:41 PM",
    "Foo": "bar baz"
  },
  "headers": {
    "Host": "httpbin.org",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Microsoft Windows 10.0.22621; en-US) PowerShell/7.2.6",
    "X-Amzn-Trace-Id": "Root=1-some-guid"
  },
  "origin": "1.2.3.4",
  // The original body was transformed into query parameters and appended to the URI
  "url": "https://httpbin.org/get?Baz=10%2F19%2F2022+3%3A23%3A41+PM&Bar=5&Foo=bar+baz"
}

[michaeltlombardi]

Update

I started doing a more thorough investigation and talked a bit with @markekraus about the implementation and it turns out I had managed to narrowly thread the needle in my initial investigation for cases where there isn't any encoding behaviors. There's actually several that need to be documented:

• How Body handles:

  1. [ ] IDictionary on non-GET calls
  2. [ ] FormObject
  3. [ ] XmlNode
  4. [ ] Stream
  5. [ ] byte[]

• [ ] Clarify that ContentType changes the encoding if specified.

I'll work to address this, but it will likely take longer than I initially hoped.

[SetTrend]

Thanks so much for taking the time to investigate on this, @michaeltlombardi!

Your efforts are very much appreciated!