pigpio will not run on a Pi 5

[sailoog]

https://github.com/joan2937/pigpio/issues/589

Investigating gpiod and pinctrl as replacement...

[sailoog]

My first search was unsuccessful, we need some library compatible with RPi5 capable of bitbanging. I'm still investigating...

[sailoog]

OK it is done :)

We already have a new python script using gpiod library instead of pigpio that works on RPi 4 and 5.

Aug 21 20:55:43 2024-08-21T19:55:43.951Z signalk-server:events:connection:seatalk:nmea0183 $STALK,26,04,1A,08,00,00,10
Aug 21 20:55:43 2024-08-21T19:55:43.976Z signalk-server:events:connection:seatalk:nmea0183 $STALK,27,01,47,01
Aug 21 20:55:43 2024-08-21T19:55:43.997Z signalk-server:events:connection:seatalk:nmea0183 $STALK,52,01,D3,00
Aug 21 20:55:44 2024-08-21T19:55:44.018Z signalk-server:events:connection:seatalk:nmea0183 $STALK,53,30,14
Aug 21 20:55:44 2024-08-21T19:55:44.044Z signalk-server:events:connection:seatalk:nmea0183 $STALK,57,00,14
Aug 21 20:55:44 2024-08-21T19:55:44.072Z signalk-server:events:connection:seatalk:nmea0183 $STALK,50,02,2A,3F,00
Aug 21 20:55:44 2024-08-21T19:55:44.099Z signalk-server:events:connection:seatalk:nmea0183 $STALK,51,02,02,49,94
Aug 21 20:55:44 2024-08-21T19:55:44.215Z signalk-server:events:connection:seatalk:nmea0183 $STALK,58,25,2A,02,7A,02,CA,D8
Aug 21 20:55:44 2024-08-21T19:55:44.315Z signalk-server:events:connection:seatalk:nmea0183 $STALK,23,01,16,48
Aug 21 20:55:44 2024-08-21T19:55:44.818Z signalk-server:events:connection:seatalk:nmea0183 $STALK,89,F2,12,00,20
Aug 21 20:55:44 2024-08-21T19:55:44.847Z signalk-server:events:connection:seatalk:nmea0183 $STALK,89,32,13,00,20
Aug 21 20:55:44 2024-08-21T19:55:44.873Z signalk-server:events:connection:seatalk:nmea0183 $STALK,10,01,00,CA
Aug 21 20:55:44 2024-08-21T19:55:44.895Z signalk-server:events:connection:seatalk:nmea0183 $STALK,11,01,11,09
Aug 21 20:55:44 2024-08-21T19:55:44.920Z signalk-server:events:connection:seatalk:nmea0183 $STALK,20,01,D0,00
Aug 21 20:55:44 2024-08-21T19:55:44.948Z signalk-server:events:connection:seatalk:nmea0183 $STALK,26,04,1C,08,00,00,10
Aug 21 20:55:44 2024-08-21T19:55:44.971Z signalk-server:events:connection:seatalk:nmea0183 $STALK,27,01,47,01
Aug 21 20:55:44 2024-08-21T19:55:44.998Z signalk-server:events:connection:seatalk:nmea0183 $STALK,52,01,D4,00
Aug 21 20:55:45 2024-08-21T19:55:45.022Z signalk-server:events:connection:seatalk:nmea0183 $STALK,53,70,14
Aug 21 20:55:45 2024-08-21T19:55:45.527Z signalk-server:events:connection:seatalk:nmea0183 $STALK,57,00,14 

The script detects the GPIO chip and uses the same UI in Signal K server. Only the embedded python code should be replaced in pigpio-seatalk.js and modify the docs. Since pigpio will no longer be used, how should we handle references to pigpio in the file name?

[tkurki]

Would it make more sense to create a new connection type rather than change the existing one? With NMEA2000 we have the data type and then source separately.

How would / should this affect the current users, with working systems? Do they keep using pigpio or are their systems silently switched over to the new way? As a user I might not want that...

[astuder]

I wrote the Python script that @sailoog tested. It works with gpiod 1.6.3 that comes preinstalled with Raspberry Pi OS Bookworm, and also with gpiod 2.2.1 that is installed when updating it with pip. I tested with Raspberry Pi OS Bookworm on Raspberry Pi 4 and 5 for both versions of gpiod.

I haven't tested with older versions of Raspberry Pi OS, or other Pi models.

Would it be possible to switch the .js silently when Bookworm and Pi 4 or Pi 5 is detected?

[astuder]

I apologize, I'm not familiar with Signal K Server.

Looking at the documentation, I can see how it may be easier to have a separate source for pigpio and gpiod. Otherwise it will be tricky to still cover the pigpio installation steps without confusing the user.

[sailoog]

Agree. The change could be made silently to the user, but adding a new source could be more consistent. If I am not wrong we should add a new field "Seatalk Source" to the Data Type "Seatalk (GPIO)" form containing something like "pigpio" and "gpiod" as options but is it possible to make it silent for existing RPi4/pigpio users without any action on their side?

For non-OpenPlotter users:

As @astuder explained, his script works with both the outdated version of gpiod installed by default on Raspberry OS, and the new active version installed with pip:

sudo pip3 install gpiod --break-system-packages

Obviously active is the recommended one, but pip will complain that the package is being installed in a non-virtual environment, but you can ignore the warning.

I can update the documentation unless @MatsA wants to do it himself.

[astuder]

So I guess a separate .js is the way to go. I will submit a pull-request for st1rx.js later today.

For the UI: How about having a new source drop-down defaulting to "gpiod", allowing the user to select "pigpio (legacy)" if they run into problems with gpiod?

I do NOT recommend updating gpiod outside of very well controlled environments. It will introduce the unsuspecting user to Python's dependency hell as any other Python application using gpiod will break. Almost all APIs changed between 1.x.x and 2.x.x. I put some effort into supporting both versions, so no need for us to push users to update gpiod.

Anyone knows how far back the Python binding of gpiod was available on Raspberry Pi? And what is realistically supported by the current version of Signal K server?

Here's the standalone Python script if anyone wants to run some tests. Please report results here. May also be interesting to see how/whether it works on Orange Pi and similar.

I plan to test with a legacy Raspian on a Raspberry Pi 3 today.

[sailoog]

https://packages.debian.org/search?keywords=gpiod

1.x.x has been there since buster and will still be there in trixie. OK then nothing about 2.x.x version in docs.

OpenPlotter has been using 2.x.x since the beginning of version 4 and no problems so far.

[astuder]

Tested on a Pi 3 with a freshly imaged Raspbian GNU/Linux 11 (bullseye): The gpiod Python library was not installed, pip installs version 2.2.1 works as expected after adding Pi 3 to the model detection

I updated the gist with that change. I also split read() into version specific implementations to improve readability, and minimize overhead.

[MatsA]

Thanks for charing the changes @sailoog . I don't fully understand the changes so would appreciate that you do the documentation update. Nice work @astuder. Thanks Mats.

[sailoog]

Thanks for charing the changes @sailoog . I don't fully understand the changes so would appreciate that you do the documentation update. Nice work @astuder. Thanks Mats.

Ok, no problem, I will try to do it today in a parallel PR.

[sailoog]

Tested on a Pi 3 with a freshly imaged Raspbian GNU/Linux 11 (bullseye): The gpiod Python library was not installed, pip installs version 2.2.1 works as expected after adding Pi 3 to the model detection

Raspbian is not the same as Raspberry OS. Raspberry OS is using the Debian repositories where gpiod was present since Buster, but Raspbian had its own repositories where gpiod was probably not present because it was not a popular library. If I am not wrong Raspbian became Raspberrry OS in bullseye but the 32bit version was still Raspbian.

It might be best to advise users to check if gpiod is installed first. If it is installed, do not touch it in case some program is using it. If not installed, install the latest version from pip.

[jimseng]

I just ran a test with the script from astuder. I'm not sure if the results are as expected. I get valid Seatalk data using my own script based around pigpiod, and seatalk_read.py (https://github.com/Thomas-GeDaD/Seatalk1-Raspi-reader). I get multiple "invalid stop bits", "invalid idle state", "not a start bit" when I run st1rx.py My Seatalk network is connected to my pi via an opto with GPIO6 pulled up to 3.3v via a 10k resistor. Just thought I would see if I can contribute to this thread as I have been dying to migrate over to my Pi 5. I have attached the output from st1rx.py ST_GPIOD_Test.txt

I am running it on a Pi 4 and had to install gpiod with pip, which installed without any issues.

[astuder]

@jimseng Thanks for testing, that's the kind of results I was looking for / feared :)

Are you able to repeat the test with a fresh Raspberry Pi image? I suspect that the issue is reliability of the solution when the Pi is under load, but it would be a good test to rule out the hardware as a cause.

@sailoog Raspian / Raspberry Pi OS: That string is what the OS reports. I don't think there was a 64bit version to choose from in the Imager.

[jimseng]

Well for no reason other than to fiddle, I changed ST_BAUD = 4900 and now it works much better. I added a socket and am sending "$STALK" + st_msg via UDP to SignalK on my Pi 5 via ethernet and am getting my instruments displaying correctly in OpenCPN. This is awesome work by astuder. I'll start wiring it to my Pi 5 shortly. I have attached the output with the baud rate set at 4900 in case anyone is interested. 4900_test.txt

[astuder]

Hmm, that's unexpected. Do you have multiple ST devices on the bus? If so, were all instruments not working, or only some?

Do you also get occasional invalid messages with pigpio? Like for example the single-byte messages.

I may be able to dynamically adjust the bit timing without too much overhead.

[astuder]

Oh, and occasional bit errors can happen with multiple instruments due to bus collisions.

I probably should make printing the bit errors a debug option. And add a mechanism to discard partial $STALK messages when bit errors are detected.

[sailoog]

The socket could be affecting performance if it is located in the loop. Could you also try the original code without modifications and run it as?:

python3 st1rx.py GPIO06 true

and

python3 st1rx.py GPIO06 false

after checking what option works better try to stop all programs or services in your system to unload the CPU and check if the result improves.

[jimseng]

I don't think the socket is the problem as I tried it purely with the original script. I didn't get expected results until I changed the baud rate. I am talking purely about looking at the printed datagrams on the screen. When I say my instruments were working in OpenCPN, I first got the datagrams to look as expected by changing the baud rate to 4900. I then moved on to adding a socket to the script to see if I could get the $STALK string over UDP to my Pi 5 (which worked straight away). For context, I read seatalk with pigpio and stream it back to a custom instrument display on my phone (plus I send autopilot commands back from my phone). That is why I know what a seatalk datagram looks like and why I recognized that it wasn't looking correct. This lead me to try changing the baud rate to 4900, for no other reason than just to try it. Immediately I recognized the datagrams and moved onto the UDP stuff. I hope this makes sense. I tried various baud rates from 4820 to 48900. Anything above 4820 produces "proper" data. I can try a fresh image, probably not tonight though. Although I am pretty sure my Pi 4 isn't running anything other than a hotspot and my ssh session. I turned off other services (apache,mysqld etc) and there was no difference. CPU running at about 5% (see screen grab of ssh sessions) Very happy to try and help. I am sailing Saturday afternoon UK time. Might have time in the morning.

[jimseng]

The screen grab above is with the baud rate set to 4900

[jimseng]

Oh and with pigpio the attached text file is the output from my own script. I don't think there are many errors, but I haven't looked that hard. pigpio_seatalk.txt

[sailoog]

OK thanks.

I have also repeated the stress test taking the Raspberry Pi 5 to 100% and I have no data loss. Even the blank space that can be seen in the Signal k log matches the Seatalk log:

Aug 23 19:25:27 2024-08-23T18:25:27.371Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,58,25,29,DE,1A,02,BC,3E
Aug 23 19:25:27 2024-08-23T18:25:27.471Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,23,01,18,4C
Aug 23 19:25:27 2024-08-23T18:25:27.970Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,89,22,0B,00,20
Aug 23 19:25:28 2024-08-23T18:25:27.999Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,89,22,0B,00,20
Aug 23 19:25:28 2024-08-23T18:25:28.024Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,10,01,02,C6
Aug 23 19:25:28 2024-08-23T18:25:28.044Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,11,01,04,03
Aug 23 19:25:28 2024-08-23T18:25:28.066Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,20,01,11,00
Aug 23 19:25:28 2024-08-23T18:25:28.097Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,26,04,A5,00,00,00,10
Aug 23 19:25:28 2024-08-23T18:25:28.117Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,27,01,5A,01
Aug 23 19:25:28 2024-08-23T18:25:28.137Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,52,01,1F,00
Aug 23 19:25:28 2024-08-23T18:25:28.156Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,53,20,0B
Aug 23 19:25:28 2024-08-23T18:25:28.470Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,57,00,14
Aug 23 19:25:28 2024-08-23T18:25:28.570Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,89,62,0B,00,20
Aug 23 19:25:28 2024-08-23T18:25:28.970Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,23,01,18,4C
Aug 23 19:25:28 2024-08-23T18:25:28.995Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,89,62,0B,00,20
Aug 23 19:25:29 2024-08-23T18:25:29.022Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,10,01,02,C5
Aug 23 19:25:29 2024-08-23T18:25:29.047Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,11,01,04,05
Aug 23 19:25:29 2024-08-23T18:25:29.067Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,20,01,11,00
Aug 23 19:25:29 2024-08-23T18:25:29.095Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,26,04,A7,00,00,00,10
Aug 23 19:25:29 2024-08-23T18:25:29.119Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,27,01,5A,01
Aug 23 19:25:29 2024-08-23T18:25:29.147Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,52,01,20,00
Aug 23 19:25:29 2024-08-23T18:25:29.173Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,53,20,0B
Aug 23 19:25:29 2024-08-23T18:25:29.198Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,57,00,14
Aug 23 19:25:29 2024-08-23T18:25:29.225Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,50,02,29,36,16
Aug 23 19:25:29 2024-08-23T18:25:29.247Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,51,02,02,D3,92
Aug 23 19:25:29 2024-08-23T18:25:29.470Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,58,25,29,DE,18,02,BC,3D
Aug 23 19:25:29 2024-08-23T18:25:29.769Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,89,A2,0B,00,20
Aug 23 19:25:29 2024-08-23T18:25:29.975Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,23,01,18,4C
Aug 23 19:25:30 2024-08-23T18:25:30.000Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,89,A2,0B,00,20
Aug 23 19:25:30 2024-08-23T18:25:30.002Z signalk-server:events:connection:seatalk1:nmea0183
Aug 23 19:25:30 2024-08-23T18:25:30.027Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,10,01,02,C5
Aug 23 19:25:30 2024-08-23T18:25:30.051Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,11,01,04,01
Aug 23 19:25:30 2024-08-23T18:25:30.071Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,20,01,11,00
Aug 23 19:25:30 2024-08-23T18:25:30.098Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,26,04,AD,00,00,00,10
Aug 23 19:25:30 2024-08-23T18:25:30.120Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,27,01,59,01
Aug 23 19:25:30 2024-08-23T18:25:30.144Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,52,01,20,00
Aug 23 19:25:30 2024-08-23T18:25:30.170Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,53,20,0B
Aug 23 19:25:30 2024-08-23T18:25:30.470Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,57,00,14
Aug 23 19:25:30 2024-08-23T18:25:30.980Z signalk-server:events:connection:seatalk1:nmea0183 $STALK,89,E2,0B,00,20 

[astuder]

Faster baudrate means that the bit gets sampled earlier, with the last bit having the biggest offset. I wonder if the spacing at the end of a byte is shorter than expected. What happens if you run at 4800 and set ST_STOP=0?

@sailoog can you run the same test with st1rx.py? Signal K may suppress non-NMEA output.

[sailoog]

Done, same result. I am sure there is no data loss because I can compare at any time what is going out from the Raymarine MFD and what is going in in the SK log or the terminal. Look at my post.

I think that could be a hardware issue, remember the problems we had when developing the HAT using slow optocouplers or the wrong resistor.

[jimseng]

ST_INVERT = 1 # 0=idle high, 1=idle low ST_BITS = 9 ST_STOP = 0 ST_BAUD = 4800

$STALK,26,04 not a start bit not a start bit not a start bit not a start bit not a start bit $STALK,FF not a start bit $STALK,27,01 not a start bit not a start bit not a start bit not a start bit not a start bit $STALK,FF not a start bit $STALK,60,0C not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit $STALK,FF not a start bit $STALK,00,02 not a start bit not a start bit not a start bit not a start bit not a start bit not a start bit

[astuder]

ah well, so much for that theory :)

[astuder]

@jimseng do you have multiple ST devices on the bus? Or just one?

[jimseng]

I have the wind, depth, speed plus a couple of St6000+ repeaters, not that I am pushing buttons.

[astuder]

Another idea: what happens if you remove halfbit_ns from this line: remaining_ns = frame_ns + halfbit_ns gpiod 1.x: https://gist.github.com/astuder/c319cf955f6d651350222d36065981b3#file-st1rx-py-L110 gpiod 2.x: https://gist.github.com/astuder/c319cf955f6d651350222d36065981b3#file-st1rx-py-L206

[jimseng]

sample_ns = e_ns + halfbit_ns remaining_ns = frame_ns #+ halfbit_ns b = 0 sb = False

Still no joy. However with the halfbit_ns commented out, 4900 baud rate still gives good results. So far that is the only thing that has made a difference. I'm trying to find an SD card. Just a thought. It is a Pi 4 but I think I am running 32 bit OS. It's been on the boat for 6 years.