libgpiod and the future of gpio in Python

[Gadgetoid]

First off, it's abundantly clear to me that - despite its deprecation - Linux sysfs GPIO (the way this library currently works) is going to stick around for quite some time, and still has uses on embedded platforms with behind-the-curve kernels. I don't want to take any functionality away from people.

However, Linux's new libgpiod is the clear path forward, and the current state of libgpio for Python users is absolutely heinous.

We have:

• The excellent but confusingly named python3-gpiod, which is a pure-python re-implementation of python3-libgpiod. It works, but is severely lacking in documentation and does not in any way attempt to be "pythonic" in its implementation- being a 1:1 mapping of the reference design. It's available via pypi but not (afaik) in any Linux package managers.
• The, effectively a reference design, python3-libgpio, this is a C library that is available via - for example - the python3-libgpiod Debian package. It does not have a pypi counterpart, so it cannot be specified as a dependency or installed in a virtual env.
• A mystery-meat libgpiod package which has been uploaded to pypi with no details or indication as to which of the above it resembles
• Third-party libraries such as RPi.GPIO (RPi only), wiringpi (defunct), and lgpio (too broad)
• Libraries like blinka or gpiozero which attempt to solve the problem of the GPIO ecosystem by sweeping it all under a more Pythonic, user-friendly API. These are inevitably too broad in scope to be a reasonable dependency for- for example- handling an interrupt pin in a simple sensor driver.
• Libraries like this one, which - in my humble opinion - represent a reasonable, Pythonic answer to the problem of GPIO, but rely on a deprecated interface.

To confound this issue, in the case of the low-level libraries (gpiod and libgpio) documentation is scant.

To confound it further, as of Debian Bookworm (the underpinnings for the new Raspberry Pi OS) it is not possible for a user to install pypi packages system-wide, or casually maintain a mix of system/user packages. All pypi packages must be installed into a virtual environment. This effectively makes python3-libgpiod impossible to specify as a dependency, since you'll end up with the user's pip pulling libgpiod (the mystery meat package mentioned above) from pypi. This is the case unless they have initialized their venv with --system-site-packages so that the system python3-libgpiod can satisfy this dependency.

I first picked up gpio (this library) because I wanted to solve this problem, and had (ha ha) mistaken this library for a libgpiod based library.

I still want to solve this problem, and - given how complicated, cryptic and confusing the various gpio package names are (lgpio, libgpio and libgpio etc) - making gpio (this library) the canonical library for Linux GPIO in Python seems like the prudent move. That means adding support for libgpiod based GPIO access into this library, and doing it right.

And, thus, this is an open call for feedback and input on this idea, how to approach it and what GPIO should look like in Python.

[vitiral]

I consider gpio to be a "system resource", therefore I don't see installing a debian package to get faster GPIO to be much of an issue. However, I think it would be better if this library works for low-speed applicaitons regardless of whether the debian package is installed.

For this library I would prefer the solutions in this order:

1. stay the way things are: this is the simplest possible solution, but if the other options are reasonably simple (low code) I will consider them.
2. have a try: import gpiod; USE_GPIOD = True; except: pass and use (aka wrap) gpiod if it's available. All of the constructor APIs would need to be able to override this boolean, and a client could override the global USE_GPIOD at their discretion (which would affect the default behavior of constructors).
3. Use a .so and interface using dynamic linking. I'm not going to personally investigate this, but if it were possible and the implementation were extremely minimal then I'd consider it.

Thoughts?

[vitiral]

https://stackoverflow.com/questions/74352978 looks like a good reference

[Gadgetoid]

Okay, plot twist- it's the author of libgpiod who uploaded libgpiod to Pypi. The package appears to be irreconcilably broken on Raspberry Pi OS (Debian Bookworm) due to a mismatch in assumed libgpiod versions.

The big problem with no 2 this is that gpiod cannot be imported from system packages while in a virtual context, unless the user specifies --system-site-packages. I'm trying to figure out how that can be fixed, since depending upon the python3-libgpiod (whether it comes via apt or pypi) seems to be most reasonable solution.

I suspect we could probably probe for the existence of /sys/class/gpio/gpiochipX and, if the library is available, prefer it.

I was broadly hoping that a widely available, widely compatible python3-libgpiod library would let us simply specify it as a dependency but it seems to be repeating the same mistakes made by smbus (that prompted the pure-python smbus2).

We might have to depend upon or largely re-implement the pure-Python alternative: https://github.com/hhk7734/python3-gpiod

[Gadgetoid]

Edit, okay here's a crude pin toggle that works with both libgpiod (official) and gpiod (pure Python)

import gpiod
import time

CONSUMER = "Benchmark"
PIN = 15


if hasattr(gpiod, "Chip"):
    chip = gpiod.Chip("/dev/gpiochip4")

    pin = chip.get_line(PIN)

    pin.request(consumer=CONSUMER, type=gpiod.LINE_REQ_DIR_OUT)

else:
    chip = gpiod.chip("/dev/gpiochip4")

    pin = chip.get_line(PIN)

    config = gpiod.line_request()
    config.consumer = "Benchmark"
    config.request_type = gpiod.line_request.DIRECTION_OUTPUT

    pin.request(config)


t_start = time.time()

n = 1000

for x in range(n):
    pin.set_value(1)
    pin.set_value(0)

t_end = time.time()

print(f"Toggling {n} times took: {(t_end - t_start) * 1000:0.4f}ms")

Results:

gpiod (pure Python):
Toggling 1000 times took: 10.7212ms
libgpiod (official):
Toggling 1000 times took: 5.5680ms

[Gadgetoid]

Okay I've got a prototype libgpiod package that I can install (via pip) on a Raspberry Pi. It - with any luck - works elsewhere too, but I'll admit I'm probing in the dark a little here.

https://github.com/Gadgetoid/libgpiod-python/releases/tag/2.0.0

I'm in the process of working out how this package could become the canonical libgpiod Python distribution, but I realize "works for me" is a very long way from clearing the bar for such an important package.

Here's the above benchmark updated for libgpiod 2.0.0 (actually 2.0.2 for reasons):

import gpiod
import time

CONSUMER = "Benchmark"
PIN = 15

chip = gpiod.Chip("/dev/gpiochip4")

lines = chip.request_lines(consumer=CONSUMER, config={
    PIN: gpiod.LineSettings(
        direction=gpiod.line.Direction.OUTPUT,
        output_value=gpiod.line.Value.INACTIVE
    ) 
})

t_start = time.time()

n = 1000

for x in range(n):
    lines.set_value(PIN, gpiod.line.Value.ACTIVE)
    lines.set_value(PIN, gpiod.line.Value.INACTIVE)

t_end = time.time()

print(f"Toggling {n} times took: {(t_end - t_start) * 1000:0.4f}ms")

[vitiral]

If you can get a gpiod installed from pip, then couldn't users simply use that?

[Gadgetoid]

If you can get a gpiod installed from pip, then couldn't users simply use that?

In theory, yes, but in practise it's another API to learn and understand to accomplish the same task. And not a terribly user-friendly one at that.

Perhaps it's hubris thinking it could be better. But it's had multiple years to make a dent in the Python ecosystem and all we've got is a bunch of stackoverflow posts with various snapshots of out of date code.

In all cases, though, it would be faster to use the library directly and that's always an option, but I very much believe there's a neat little gap between libgpiod and gpiozero where a very concise GPIO library can sit for the benefit of people who aren't either engineers or kids/beginners respectively.

Also gives a migration path for those using this library to transition painlessly over to libgpiod if/when they get rugged by their distro 😆

[vitiral]

This does sound concerning, but I don't think that making this library more complicated is going to help anyone. In fact, folks are better helped by this library being stable and constant IMO.

PyPi's fragmented ecosystem is outside the scope of this package :smile: