node-red-node-pi-gpio and node-red-contrib-python3-function don't work on Ubuntu 20.10

[votecoffee]

Which node are you reporting an issue on?

node-red-node-pi-gpio node-red-contrib-python3-function

What are the steps to reproduce?

Using image ubuntu-20.10-preinstalled-desktop-arm64+raspi.img.xz sudo apt update sudo apt upgrade sudo apt full-upgrade

sudo rpi-eeprom-update #CURRENT: Thu 03 Sep 2020 12:11:43 PM UTC (1599135103)

#GPIO support sudo groupadd -f --system gpio sudo groupadd -f --system i2c sudo groupadd -f --system spi sudo groupadd -f --system input sudo usermod -a -G gpio root sudo usermod -a -G i2c root sudo usermod -a -G spi root sudo usermod -a -G input root sudo usermod -a -G gpio user sudo usermod -a -G i2c user sudo usermod -a -G spi user sudo usermod -a -G input user sudo chown root:gpio /dev/gpiomem #use 99-com.rules from rasbian OS #99-com.rules

SUBSYSTEM=="input", GROUP="input", MODE="0660"
SUBSYSTEM=="i2c-dev", GROUP="i2c", MODE="0660"
SUBSYSTEM=="spidev", GROUP="spi", MODE="0660"
SUBSYSTEM=="bcm2835-gpiomem", GROUP="gpio", MODE="0660"

SUBSYSTEM=="gpio", GROUP="gpio", MODE="0660"
SUBSYSTEM=="gpio*", PROGRAM="/bin/sh -c '\
        chown -R root:gpio /sys/class/gpio && chmod -R 770 /sys/class/gpio;\
        chown -R root:gpio /sys/devices/virtual/gpio && chmod -R 770 /sys/devices/virtual/gpio;\
        chown -R root:gpio /sys$devpath && chmod -R 770 /sys$devpath\
'"

KERNEL=="ttyAMA[01]", PROGRAM="/bin/sh -c '\
        ALIASES=/proc/device-tree/aliases; \
        if cmp -s $ALIASES/uart0 $ALIASES/serial0; then \
                echo 0;\
        elif cmp -s $ALIASES/uart0 $ALIASES/serial1; then \
                echo 1; \
        else \
                exit 1; \
        fi\
'", SYMLINK+="serial%c"

KERNEL=="ttyS0", PROGRAM="/bin/sh -c '\
        ALIASES=/proc/device-tree/aliases; \
        if cmp -s $ALIASES/uart1 $ALIASES/serial0; then \
                echo 0; \
        elif cmp -s $ALIASES/uart1 $ALIASES/serial1; then \
                echo 1; \
        else \
                exit 1; \
        fi \

sudo mv 99-com.rules /etc/udev/rules.d sudo udevadm control --reload-rules && sudo udevadm trigger

#misc sudo adp intsall -y curl

#Python3 sudo apt install -y python3 python3-dev sudo apt install -y python3-pip

#Python3 GPIO sudo apt install -y python3-rpi.gpio sudo python3 -m pip install gpiozero

#I verfied using PyCharm that both RPi.GPIO and gpiozero import and work at this step

#NodeRED #https://nodered.org/docs/getting-started/raspberrypi sudo apt install -y build-essential git sudo apt-get install nodejs sudo apt install -y npm sudo npm install -g --unsafe-perm node-red sudo npm install -g --unsafe-perm node-red node-red-admin sudo npm install -g --unsafe-perm node-red-contrib-python3-function sudo npm install -g --unsafe-perm node-red-contrib-uibuilder sudo npm install -g --unsafe-perm node-red-dashboard sudo npm install -g --unsafe-perm node-red-node-pi-gpio sudo npm install -g --unsafe-perm node-red-node-random sudo npm install -g --unsafe-perm node-red-node-ping sudo npm install -g --unsafe-perm node-red-contrib-play-audio sudo npm install -g --unsafe-perm node-red-node-smooth sudo npm install -g --unsafe-perm node-red-node-serialport npm config set update-notifier false npm rebuild --unsafe-perm sudo ufw allow 1880 #In /usr/bin, copy python3 and rename as python. Otherwise python3 and GPIO won't work! node-red

#Receive the error: #/bin/sh: 1: /usr/local/lib/node_modules/node-red-node-pi-gpio/testgpio.py: not found

#Create GPIO input and output nodes via webpage #Deploy flow, test. Does not work.

#Create Python3 Script node via webpage #Deploy flow, test. Does not work, crashes NodeRed.

What happens?

RPi-GPIO module does not load. IO fails to work. See N/A on node status text. Python3 script causes NodeRed to crash/shutdown

What do you expect to happen?

I expected the modules to load/work

Please tell us about your environment:

• [x] Node-RED version: 1.2.6
• [x] node.js version: 14.15.1
• [x] npm version: 6.4.8
• [x] Platform/OS: Raspberry Pi 4Model B 4GB firmware version 1599135103, ubuntu-20.10-preinstalled-desktop-arm64+raspi.img.xz (problem also occurs in UbuntuMate 20.04 Raspberry Pi image)
• [x] Browser: FireFox

The root cause is that there is no python command, only python3. node-red-node-pi-gpio and node-red-contrib-python3-function both appear to reference python and assume it will map to python3, but this isn't true in most cases. I can fix the issue by going into /usr/bin, copying python3 to the filename python. After that, everything works as expected.

[votecoffee]

My issue is solved from my end, I'm looking for the nodes to be fixed however.

[knolleary]

Hi @votecoffee

please fill in the issue template so we have some idea of what the actual problem you're reporting is. Otherwise we have no information as to what your problem is and we may as well close this issue.

Also node-red-contrib-python3-function is not a module we maintain, so you'd have to report the issue against its repository.

[votecoffee]

I fixed it. I had left the comment bit at the beginning

[dceejay]

Closing here as you have opened over here instead - https://github.com/node-red/linux-installers/issues/11 thanks

[knolleary]

There are different issues. I transferred the other one that was opened in this repo originally over to the linux-installer repo.

(Unless you've already identified a common route cause... )

[votecoffee]

Yes, these are different issues and do not have a common cause that I can see. This issue has a workaround that I mentioned and I wanted to let you guys know so you could see if there was a patch that could be applied to handle a change in how python.

[dceejay]

Happy to accept help on this as rather busy with the day job at the moment. Though you say in the issue the root cause is that we call python rather than python3. Given that python3 is now the default if we replaced our call to python with python3 then would that fix it ?

[votecoffee]

That would fix it, but wouldn't support backwards compatibility. I liked the idea of creating a simlink for python if it doesn't exist. That offers good backwards compatibility.

[votecoffee]

I'm in no rush. I jury rigged it on my end so it's working and not holding me up.

[hardillb]

If you want to do that you should look at using the /etc/alternatives system to set it up

https://www.redhat.com/sysadmin/alternatives-command

[dceejay]

but would not all systems now have python3 available (even if they don't link it to python - as python (2) was EOL back in January. (so yes a major version bump for this package to drop support) but meh...?!? thoughts ?

We could of course (and may well) fix the install script to check and symlink it - but ideally that should be done by the node install - not the script as that would then work on systems that don't run the script... BUT the symlink needs to be created as root and we don't really want to install this node as root etc... just on the offchance that a postinstall script may need to create a symlink.

[MichaIng]

As we also face this issue now, after migrating all software to Python 3, I want to give this a little push.

A Node module cannot and must not provide a python command, alternative or direct symlink, as long as it does not ship the Python 2 stack behind it. That would only cause havoc on systems when real Python 2 is (still) installed for whatever reason, and is reasonably not possible anyway without root permissions.

To not break backwards compatibility, the command call needs to be done conditionally. Logic would be to prefer the python3 command, when available, else fall back to python (2). But practically I see there are a lot of direct python (2) calls in the code in this repo and don't know which one belongs to node-red-node-pi-gpio, so not sure how to implement best.

---

Having a look through the python calls:

In shell scripts, something like this would work:

python_cmd='python3'
# Fallback to Python 2, if Python 3 is not available
command -v python3 > /dev/null || python_cmd=`python`

$python_cmd -c "import sense_hat" 2> /dev/null

But since the script's aim is to check whether a library is available, the Python version choice actually needs to be done in the parent calling script and the check needs to check for the Python version the parent script has chosen, e.g. by passing it as argument?

Then we have Python scripts with shebangs: https://github.com/node-red/node-red-nodes/blob/master/hardware/unicorn/unihat.py First of all, to allow custom compiled Python binaries and custom PATHs, #!/usr/bin/env python should be used. I don't know a way to have a shebang with python3 preference and python (2) fallback, but it only matters when executing directly from shell, so should not cause issues here.

But this JavaScript should be the culprit here? Probably this would work:

        else if (execSync('python3 -c "import rpi_ws281x"').toString() !== "" && execSync('python -c "import rpi_ws281x"').toString() !== "") {
            RED.log.warn("rpi-neopixels : Can't find neopixel python library");
            allOK = false;
        }

But it is a bit similar with the first case: It does not make sense to check for a library for a specific Python version or allowing both without knowing which version will be finally used. Same problem here: https://github.com/node-red/node-red-nodes/blob/master/hardware/neopixel/scripts/checklib.js

Probably the used Python version needs to be estimated somewhere in the main scripts and then passed as environment variable or command argument for those checks to be done correctly. Whether the everything does actually work with Python 3 or not (if the checked libraries do exist for Python 3), is another question which can then be tested.

[dceejay]

I think most nodes we have that do this that call python do so via a shell script so I think you first suggestion will fix the majority of cases. Though at what point will the alias python refer to version 3 by default ?

[MichaIng]

Though at what point will the alias python refer to version 3 by default ?

Good question. At least I haven't seen this being the case on any OS yet. Instead, the python command is not served at all on Debian Bullseye and Ubuntu hirsute. They don't ship a python package anymore, but a python2 package instead and nearly no additional Python 2 module packages. It ships the python2.7 and python2 commands, but no python command, neither does any Python 3 package do. But not sure how other distros handle it.

[dceejay]

so really we could just default to python3 everywhere

[MichaIng]

Would break backwards-compatibility with Python 2, of course. But as of Python 2 deprecation it's probably not worth the effort to keep backwards compatibility.

But I guess some testing and/or transition time for developers would be required to assure that everything works well with Python 3?

[dceejay]

Well when we move to Node-RED 2 and drop node8 and 10 then they are the most likely systems still to be running python2. - and so will need updating anyway. But yes - I have added your simple bash test for now.