edit icon indicating copy to clipboard operation
edit copied to clipboard

Strange encoding

Open matsmcp opened this issue 6 months ago • 9 comments

Getting a strange encoding on a powershell script real code [CmdletBinding()] Param( in edit

Image

matsmcp avatar May 20 '25 08:05 matsmcp

Can you use the PowerShell Format-Hex tool to decode your file so we can see what bytes your file actually contains? It may be UTF-8, BOM or some other encoding within that is not being interpreted correctly.

rhubarb-geek-nz avatar May 20 '25 16:05 rhubarb-geek-nz

I, too, would love to know more.

DHowett avatar May 20 '25 21:05 DHowett

In edit

Image

in powershell

Image

and in ISE

Image

matsmcp avatar May 21 '25 05:05 matsmcp

Your file has standalone carriage return characters, 0D, rather than the traditional line feed only on POSIX or carriage return followed by line feed on Windows/DOS/OS2.

I would expect either just 0A or 0D 0A, not 0D alone.

Classic MacOS prior to MacOS X used carriage returns by themselves, but I don't think PowerShell has been ported to that.

For testers, use the following PowerShell to recreate

"[CmdletBinding()]`rParam(`r" | Set-Content 'test.ps1' -NoNewline

rhubarb-geek-nz avatar May 21 '25 06:05 rhubarb-geek-nz

Strange indeed. The code was written in ISE on a 19 or 22 server. Both ISE and powershell as a shell reads and runs the code

matsmcp avatar May 21 '25 07:05 matsmcp

I made the decision to not support CRs (carriage returns) as line breaks because that simplifies seeking across lines a lot. At some point people will probably ask for support for it, or for the various other line break types that Unicode supports, but I personally am very adamant about not wanting to add support for anything but LF and CRLF. It makes the editor design a lot easier.

lhecker avatar May 21 '25 12:05 lhecker

Put me in as number one on that request :)

I want to be able to use Edit to make changes to Powershell scripts over SSH. For that to work reliably Edit needs to understand the formats that Powershell (ISE) may use

matsmcp avatar May 21 '25 18:05 matsmcp

It isn't a format I would encourage, git does not like it at all

PS> git diff
diff --git a/test.ps1 b/test.ps1
index b8881a9..16479aa 100644
--- a/test.ps1
+++ b/test.ps1
@@ -1 +1 @@
-[CmdletBinding()]^MParam(
\ No newline at end of file
+[CmdletBinding()]^MPaXram(^M
\ No newline at end of file

It does not treat them as newlines, but as a carriage return ^M

rhubarb-geek-nz avatar May 21 '25 19:05 rhubarb-geek-nz

I just tested this with ISE on Windows 11 and it produces a UTF8-with-BOM document with CRLF line endings. To my knowledge ISE hasn't been updated in years (a decade?) so that can't be it.

lhecker avatar May 27 '25 19:05 lhecker