astro4j icon indicating copy to clipboard operation
astro4j copied to clipboard

ImageMath script: Byte Order Mark issue

Open vnp85 opened this issue 9 months ago • 9 comments

I ran into this BOM:

windows 11, JSol'Ex 2.11.3 (dev), .math generated via borges (my own script) then edited with notepad

Image

I guess my script and/or notepad though it could be funny to add a BOM, and perhaps break JSol'Ex

vnp85 avatar Mar 19 '25 15:03 vnp85

hex view with the BOM, first two bytes, FF FE after editing them out in JSolEx and saving the .math to disk

Image

vnp85 avatar Mar 19 '25 15:03 vnp85

PS: I made sure my script outputs ASCII but others may also run into weird stuff like this.

vnp85 avatar Mar 19 '25 16:03 vnp85

The real question is: why folks are using Windows :)

Joke aside, JSol'Ex expects the file to be encoded in UTF_8, but it will do best effort to find other encodings.

melix avatar Mar 19 '25 16:03 melix

"The real question is: why folks are using Windows :)" I totally agree, I am sorry and I repent. Mainly vendor lock-in induced comfort, with all the software at hand (or hand written), and without going full rabid on the issue, at the moment it is only microsoft who keeps ruining stuff that used to work in the past. I began with assembly (why x86 I hear you ask :P :P )

Well, JSolEx goes pretty haywire with UTF16, as shown by the lack of coloring (syntax highlight), even though edited out the BOM.

I think it might have happened while piping a console output of nodejs into the .math file, in VSC's (I repent, again) terminal... I rewrote borges to write the file through nodejs, with a forced encoding, circumventing whatever there was in the pipe. (in the past I copy-pasted into a notepad, but since notepad got the AI)

I repent :D

vnp85 avatar Mar 19 '25 16:03 vnp85

PS: we had an issue like this in a production project, with a json file that somehow got a BOM in front of the curly. It is beyond me how that happened, but it was a pretty deep rabbit hole, because visually it was the most innocent of JSONs that still broke the entire thing: { "foo": "bar" }

vnp85 avatar Mar 19 '25 16:03 vnp85

Don't repent. I am using Windows for captures, because, you know, Sharpcap and N.I.N.A. Yet, for daily work and processing, Linux is in my heart :)

melix avatar Mar 19 '25 16:03 melix

At work is a work story, but with astro... In 2019 I wrote my planetary autoguider, the soapbox project, that grew into the Ersatz-Obsi control panel.

That -- through the winapi -- looks into the pixelcanvas of sharpcap to reason about what it sees, without myself implementing a camera driver, or dealing with sharing the same camera resource etc...

That's how now the shg auto-capture is working, it monitors rising and falling edges in the spectrum images's luminosity, to know when to stop recording and how when the slewing is ready for the next recording to start. I had lunch outside of the building while the SHG was zig-zagging the sky. One reason I didn't switch and remained stuck with this abomination of an OS.

vnp85 avatar Mar 19 '25 16:03 vnp85

I am definitely thinking about a way to "recenter" and automatically start/stop captures in Sharpcap, but the Python API doesn't provide access to the current frames for analysis as far as I can tell. But that would be super useful. I know you wrote a custom driver, would be great to have something working on top of eqmod.

melix avatar Mar 19 '25 16:03 melix

This is a bit off topic, under the BOM, but this is my setup, and it is really obvious that there is more to it than an eqmod driver. An entire orchestrator system is needed, especially if you run into a weird bug that, through winapi I could send simple key codes to sharpcap, but not the complex ones, hence I threw in an embedded device masquerading as a keyboard, and it works.

Image

vnp85 avatar Mar 19 '25 20:03 vnp85