Memory Leak possibly with gpio or piface inputs/outputs

[griesi]

Describe the bug RAM consumption of mqtt-io process is growing over time

Expected behavior RAM consumption is on a constant level

Error messages and traceback no errors but raspberry pi is freezing when out of memory

Config

mqtt:
  host: smarthome.local
  port: 8883
  user: mqtt-heaterpi
  topic_prefix: heaterpi/piface
  tls:
    enabled: true
    ca_certs: /home/pi/mqtt_io/iobroker_ca.crt
    certfile: /home/pi/mqtt_io/mqtt-heaterpi.crt
    keyfile: /home/pi/mqtt_io/mqtt-heaterpi.key
gpio_modules:
  - name: piface2
    module: piface2
digital_inputs:
  - name: input_1
    module: piface2
    pin: 0
  - name: input_2
    module: piface2
    pin: 1
  - name: input_3
    module: piface2
    pin: 2
  - name: input_4
    module: piface2
    pin: 3
  - name: input_5
    module: piface2
    pin: 4
  - name: input_6
    module: piface2
    pin: 5
  - name: input_7
    module: piface2
    pin: 6
  - name: input_8
    module: piface2
    pin: 7
digital_outputs:
  - name: output_1
    module: piface2
    pin: 0
  - name: output_2
    module: piface2
    pin: 1
  - name: output_3
    module: piface2
    pin: 2
  - name: output_4
    module: piface2
    pin: 3
  - name: output_5
    module: piface2
    pin: 4
  - name: output_6
    module: piface2
    pin: 5
  - name: output_7
    module: piface2
    pin: 6
  - name: output_8
    module: piface2
    pin: 7

Hardware

• Platform: Raspberry Pi 1B / Raspberry Pi 2B
• Connected hardware: PiFace 1 Board / PiFace 2 Board

System:

• OS: lates Raspberry Pi OS 11.2
• Python version: 3.9.2
• User you're running as: root
• Using a virtualenv?: no

Additional context See also

• https://github.com/flyte/mqtt-io/issues/236#issuecomment-934574445
• https://github.com/flyte/mqtt-io/issues/236#issuecomment-1036012818
• https://github.com/flyte/mqtt-io/issues/236#issuecomment-1079652027

I worked around this by restarting the process once a day by the supervisor daemon which is controlling the process.

[Bettman66]

use the old 0.5.6 Version, it works great.

[griesi]

use the old 0.5.6 Version, it works great.

pip install mqtt-io==0.5.6

says that this version does not exist.

Can you please tell me which package/version you mean?

• mqtt-io
• asyncio-mqtt
• paho-mqtt
• pifacecommon
• pifacedigitalio
• gpiozero (which I am NOT using)
• RPi.GPIO (which I am NOT using)
• supervisor

[Bettman66]

https://github.com/flyte/mqtt-io/tags?after=2.0.0

or my fork

sudo apt install python3.9-distutils python3-pip git

sudo pip install setuptools

git clone https://github.com/....

cd mqt*

sudo python3 setup.py install

[callifo]

Can confirm same issue,

Config: Running in a venv, with python 3.9.2

mqtt:
  host: localhost
  status_payload_running: available
  status_payload_stopped: unavailable
  status_payload_dead: unavailable
  topic_prefix: gpio
  port: 1883
  user: "pi-mqtt-gpio"
  password: "xxx"
  protocol: 3.1.1
  ha_discovery:
    enabled: yes

gpio_modules:
  - name: raspberrypi
    module: raspberrypi
    cleanup: yes

digital_outputs:
  - name: nightlight_1
    module: raspberrypi
    pin: 14
    retain: yes
    ha_discovery:
      component: switch
      name: "Night Light 1"
  - name: nightlight_2
    module: raspberrypi
    pin: 15
    retain: yes
    ha_discovery:
      component: switch
      name: "Night Light 2"
{repeat 4 more times}

digital_inputs:
  - name: motionsensor_1
    module: raspberrypi
    pin: 0
    pullup: yes
    pulldown: no
    retain: yes
    inverted: true
    ha_discovery:
      component: binary_sensor
      name: "Motion Sensor 1"
      device_class: motion
  - name: motionsensor_2
    module: raspberrypi
    pin: 5
    pullup: yes
    pulldown: no
    retain: yes
    inverted: true
    ha_discovery:
      component: binary_sensor
      name: "Motion Sensor 2"
      device_class: motion
{repeat 10 more times}

[pakagi]

Hello,

I observe the memory leak too, but what is even worse in my case: I see the cpu usage increasing too. It starts on some 2%, but after about one day it gets up to about 100%, and after some more time it even seems to not catch up it's work and some sensor values start to not be reported occasionally (shown then as "unavailable" in HA). Restart of mqttio helps then (but that has also some other negative consequences on gpio outputs temporarily switched to unexpected state, so I cannot afford doing anything like cron based mqttio preventive restarts, without coordinating with the rest of system depending on that..)

I tried to find if it is related to any particular module or sensor, by commenting out the config sections one by one for all I use (rpi gpio out, in, ds1820 temp sensor, ads115 voltage adc) and watching resident memory and cpu usage after 5min and after 10mion after restart. But I was not able to find particular module that would be causing that - the only case when it is not happenning is when mqttio config is completely empty :-(

I was observing this on 2.2.6, running on Rpi 4, but now retried on 2.2.7 and still the same.

Pavel

[richardcastle]

use the old 0.5.6 Version, it works great.

Seems this build suffers the same leak if my testing this morning is correct unfortunately.... [EDIT - I was wrong - see below]

[Bettman66]

I use this Tree since three years without problems.

[richardcastle]

Indeed you are correct. I hadn't completely removed the latest version, so despite installing an older version, I was still running up the latest version which has got the memory leak issue.

0.5.6 appears to work fine with GPIO status being sent over MQTT - memory appears to be stable too.

Those needing to do this whilst this bug is being worked on will need to follow the instructions for that specific build as the syntax is different from the latest version!

[BenjiU]

This should be fixed with version 2.2.8 Please try and report. Thanks

[pakagi]

On my setup, after one day, seems ok. I can remove cron task to restart it daily :-) Thx !

[BenjiU]

Fixed with v2.2.8