[HELP] Install thingsboard-gateway v3.1 in Raspberry Pi

[alexcorvis84]

Describe the issue I'm trying to install thingsboard-gateway v3.1 in Raspberry Pi in following the deb installation

After issuing sudo apt install ./python3-thingsboard-gateway.deb -y I'm getting the following message:

How can I install the missing dependencies?

Versions (please complete the following information):

• OS: [Raspbian GNU/Linux 9 (stretch)]
• Thingsboard IoT Gateway version [3.1]
• Python version[3.5.3]

[samson0v]

Hi @alexcorvis84, I think it can help you:

Step 1:

export GRPC_BUILD_WITH_BORING_SSL_ASM=false
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true
export GRPC_PYTHON_BUILD_WITH_CYTHON=true
export GRPC_PYTHON_DISABLE_LIBC_COMPATIBILITY=true
export GRPC_PYTHON_LDFLAGS="-lpthread -Wl,-wrap,memcpy -static-libgcc"

Step 2:

pip3 install --upgrade grpcio-tools --force-reinstall --no-binary=grpcio-tools
pip3 install --upgrade grpcio --force-reinstall --no-binary=grpcio

Step 2 took a long time and was sometimes never completed (Raspberry Pi 3). So I did next short cut solution: Uninstall grpcio;

pip uninstall grpcio 
pip uninstall grpcio-status 

Then,

pip install grpcio==1.44.0 --no-binary=grpcio 
pip install grpcio-tools==1.44.0 --no-binary=grpcio-tools

[alexcorvis84]

Hi @alexcorvis84, I think it can help you:

Step 1:

export GRPC_BUILD_WITH_BORING_SSL_ASM=false
export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=true
export GRPC_PYTHON_BUILD_WITH_CYTHON=true
export GRPC_PYTHON_DISABLE_LIBC_COMPATIBILITY=true
export GRPC_PYTHON_LDFLAGS="-lpthread -Wl,-wrap,memcpy -static-libgcc"

Step 2:

pip3 install --upgrade grpcio-tools --force-reinstall --no-binary=grpcio-tools
pip3 install --upgrade grpcio --force-reinstall --no-binary=grpcio

Step 2 took a long time and was sometimes never completed (Raspberry Pi 3). So I did next short cut solution: Uninstall grpcio;

pip uninstall grpcio 
pip uninstall grpcio-status 

Then,

pip install grpcio==1.44.0 --no-binary=grpcio 
pip install grpcio-tools==1.44.0 --no-binary=grpcio-tools

Hi @samson0v as far as I understand these instructions are aimed if you follow the installation from pip. I already found them published by santobedi user.

BUT I'm following the installation as DEB package

This BUG needs to be addressed somehow and make the installation for the users as easy as it was in the previous versions, without the need to deal with the installation of external dependencies or at least update the documentation accordingly with the official necessary steps. Don't you think?

With the latest versions, is this the normal behavior when installing?

[samson0v]

@alexcorvis84 I think this instruction can be also useful when you install via the deb package because of system dependencies. Also, we will remake the deb installation as it was in the previous versions. Sorry for the inconvenience.

[alexcorvis84]

@alexcorvis84 I think this instruction can be also useful when you install via the deb package because of system dependencies.

Not sure about it. It's just exporting environmental variables and updating pip packages that would help to the installation from from pip ONLY.

Also, we will remake the deb installation as it was in the previous versions. Sorry for the inconvenience.

Yes, please. I think this need to be addressed as soon as possible as several users have already been affected by this disruptive change that complicates the installation ;-) Please keep me updated once remake has been done! Thanks in advance!

[samson0v]

Hi @alexcorvis84, apt source seems to be outdated and there is no longer this package on the repo for this version of the OS, this usually happens when a new version of the system is released. The fix is either installing with pip or installing the latest version of the OS, which includes the latest repositories, or manually adding a repository with these packages for this version of the OS.

[alexcorvis84]

Hi @alexcorvis84, apt source seems to be outdated and there is no longer this package on the repo for this version of the OS, this usually happens when a new version of the system is released. The fix is either installing with pip or installing the latest version of the OS, which includes the latest repositories, or manually adding a repository with these packages for this version of the OS.

I'll try to update the OS packages on device to latest versions and try again the .deb installation. I'll keep you updated with the results

[alexcorvis84]

Update: I did an apt update & apt upgrade on device and still facing the same unmet dependencies message

[imbeacon]

@alexcorvis84 ,

This issue doesn't relate to the installation of the gateway, it actually means that there is no package on apt repository for your system, but it is still present in pip, so you can do install via pip using sudo and only after this install the deb package.

[alexcorvis84]

@alexcorvis84 ,

This issue doesn't relate to the installation of the gateway, it actually means that there is no package on apt repository for your system, but it is still present in pip, so you can do install via pip using sudo and only after this install the deb package.

Hi @imbeacon

I already did (installed missing dependencies via pip), but after issuing again the command to install the .deb package I'm still getting the same message. Please check below image.

[imbeacon]

Please try to run sudo apt -f install ./python3-thingsboard-gateway.deb

[alexcorvis84]

Please try to run sudo apt -f install ./python3-thingsboard-gateway.deb

I did and got the same message as before.

[imbeacon]

Usually it helps with this problem and solves broken packages. You can use the following command as a workaround*: sudo dpkg -i --ignore-depends=python3-paho-mqtt,python3-grpcio python3-thingsboard-gateway.deb , because you already have installed packages. Another way is try to use apt with -f -m options.

[alexcorvis84]

Usually it helps with this problem and solves broken packages. You can use the following command as a workaround*: sudo dpkg -i --ignore-depends=python3-paho-mqtt,python3-grpcio python3-thingsboard-gateway.deb , because you already have installed packages. Another way is try to use apt with -f -m options.

That way didn't help. It throws even more dependencies are needed.

[imbeacon]

Unfortunately it means the same - you cannot get these packages from the repository. To fix this, you need to change source file for apt by adding the mirror to get the packages. How to change the mirror for raspbian

[alexcorvis84]

Unfortunately it means the same - you cannot get these packages from the repository. To fix this, you need to change source file for apt by adding the mirror to get the packages. How to change the mirror for raspbian

Thanks @imbeacon for feedback and suggestion.

I think the shortest way to avoid these problems is upgrading the RaspberryPi OS to a newer version. Currently this device is using the "Stretch" version (which is a bit outdated) and I'm facing these dependencies loop problem, however, on another RaspberryPi I have with "Buster" I don't remember having any problems with dependencies, running .deb installation smoothly. I found this article "How to Upgrade Raspbian Stretch to Raspbian Buster" that might help me.

I wonder if you have tested with RaspberryPi (latest release) and what version of operating system you used?

Thanks in advance.

[imbeacon]

@alexcorvis84 You are right, the simplest way is updating the system to the latest version. But if you have some necessary data- you can just change mirrors for the repository. During testing we are using the latest version of the OS, available at the moment of the testing.

[alexcorvis84]

@alexcorvis84 You are right, the simplest way is updating the system to the latest version. But if you have some necessary data- you can just change mirrors for the repository. During testing we are using the latest version of the OS, available at the moment of the testing.

So I'll try to update OS. I'll keep you updated once tested.

[samson0v]

Hi @alexcorvis84, any updates?

[alexcorvis84]

Hi @alexcorvis84, any updates?

Hi @samson0v, I've not been able to perform the test yet. Hope to get an update along this week. I'll keep you posted. Thanks in advance!

Update: fresh new install SD running latest Raspberry Pi OS (with Python 3.9.7 installed)

Once issued "sudo apt install ./python3-thingsboard-gateway.deb -y" installation Failed.

See Logs:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'python3-thingsboard-gateway' instead of './python3-thingsboard-gateway.deb'
The following additional packages will be installed:
  figlet libc-ares2 libprotobuf17 python3-cachetools python3-decorator python3-grpcio python3-jsonpath-rw python3-paho-mqtt python3-ply python3-protobuf python3-pyfiglet python3-regex
  python3-simplejson python3-termcolor python3-yaml toilet-fonts
Suggested packages:
  python-ply-doc toilet
The following NEW packages will be installed:
  figlet libc-ares2 libprotobuf17 python3-cachetools python3-decorator python3-grpcio python3-jsonpath-rw python3-paho-mqtt python3-ply python3-protobuf python3-pyfiglet python3-regex
  python3-simplejson python3-termcolor python3-thingsboard-gateway python3-yaml toilet-fonts
0 upgraded, 17 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/3,413 kB of archives.
After this operation, 14.3 MB of additional disk space will be used.
Get:1 /home/ansible/python3-thingsboard-gateway.deb python3-thingsboard-gateway all 3.1-1 [118 kB]
Traceback (most recent call last):               
  File "/usr/bin/apt-listchanges", line 30, in <module>
    import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'
Selecting previously unselected package figlet.
(Reading database ... 52648 files and directories currently installed.)
Preparing to unpack .../00-figlet_2.2.5-3_armhf.deb ...
Unpacking figlet (2.2.5-3) ...
Selecting previously unselected package libc-ares2:armhf.
Preparing to unpack .../01-libc-ares2_1.14.0-1+deb10u1_armhf.deb ...
Unpacking libc-ares2:armhf (1.14.0-1+deb10u1) ...
Selecting previously unselected package libprotobuf17:armhf.
Preparing to unpack .../02-libprotobuf17_3.6.1.3-2+rpi1_armhf.deb ...
Unpacking libprotobuf17:armhf (3.6.1.3-2+rpi1) ...
Selecting previously unselected package python3-cachetools.
Preparing to unpack .../03-python3-cachetools_3.1.0-2_all.deb ...
Unpacking python3-cachetools (3.1.0-2) ...
Selecting previously unselected package python3-decorator.
Preparing to unpack .../04-python3-decorator_4.3.0-1.1_all.deb ...
Unpacking python3-decorator (4.3.0-1.1) ...
Selecting previously unselected package python3-grpcio.
Preparing to unpack .../05-python3-grpcio_1.16.1-1_armhf.deb ...
Unpacking python3-grpcio (1.16.1-1) ...
Selecting previously unselected package python3-ply.
Preparing to unpack .../06-python3-ply_3.11-3_all.deb ...
Unpacking python3-ply (3.11-3) ...
Selecting previously unselected package python3-jsonpath-rw.
Preparing to unpack .../07-python3-jsonpath-rw_1.4.0-3_all.deb ...
Unpacking python3-jsonpath-rw (1.4.0-3) ...
Selecting previously unselected package python3-paho-mqtt.
Preparing to unpack .../08-python3-paho-mqtt_1.4.0-1_all.deb ...
Unpacking python3-paho-mqtt (1.4.0-1) ...
Selecting previously unselected package python3-protobuf.
Preparing to unpack .../09-python3-protobuf_3.6.1.3-2+rpi1_armhf.deb ...
Unpacking python3-protobuf (3.6.1.3-2+rpi1) ...
Selecting previously unselected package python3-pyfiglet.
Preparing to unpack .../10-python3-pyfiglet_0.7.4+dfsg-3_all.deb ...
Unpacking python3-pyfiglet (0.7.4+dfsg-3) ...
Selecting previously unselected package python3-regex.
Preparing to unpack .../11-python3-regex_0.1.20190207-1_armhf.deb ...
Unpacking python3-regex (0.1.20190207-1) ...
Selecting previously unselected package python3-simplejson.
Preparing to unpack .../12-python3-simplejson_3.16.0-1_armhf.deb ...
Unpacking python3-simplejson (3.16.0-1) ...
Selecting previously unselected package python3-termcolor.
Preparing to unpack .../13-python3-termcolor_1.1.0-2_all.deb ...
Unpacking python3-termcolor (1.1.0-2) ...
Selecting previously unselected package python3-yaml.
Preparing to unpack .../14-python3-yaml_3.13-2_armhf.deb ...
Unpacking python3-yaml (3.13-2) ...
Selecting previously unselected package python3-thingsboard-gateway.
Preparing to unpack .../15-python3-thingsboard-gateway.deb ...
Installing directory for configs...
Warning: The home dir /var/lib/thingsboard_gateway you specified already exists.
The system user `thingsboard_gateway' already exists. Exiting.
mkdir: cannot create directory ‘/var/lib/thingsboard_gateway/extensions’: File exists

Unpacking python3-thingsboard-gateway (3.1-1) ...
Selecting previously unselected package toilet-fonts.
Preparing to unpack .../16-toilet-fonts_0.3-1.2_all.deb ...
Unpacking toilet-fonts (0.3-1.2) ...
Setting up python3-termcolor (1.1.0-2) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up toilet-fonts (0.3-1.2) ...
Setting up python3-cachetools (3.1.0-2) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up python3-ply (3.11-3) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up python3-yaml (3.13-2) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up libc-ares2:armhf (1.14.0-1+deb10u1) ...
Setting up python3-simplejson (3.16.0-1) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up python3-decorator (4.3.0-1.1) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up python3-grpcio (1.16.1-1) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up python3-paho-mqtt (1.4.0-1) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up figlet (2.2.5-3) ...
update-alternatives: using /usr/bin/figlet-figlet to provide /usr/bin/figlet (figlet) in auto mode
Setting up python3-regex (0.1.20190207-1) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up python3-pyfiglet (0.7.4+dfsg-3) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up libprotobuf17:armhf (3.6.1.3-2+rpi1) ...
Setting up python3-protobuf (3.6.1.3-2+rpi1) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up python3-jsonpath-rw (1.4.0-3) ...
update-alternatives: using /usr/bin/python3-jsonpath to provide /usr/bin/jsonpath (jsonpath) in auto mode
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Setting up python3-thingsboard-gateway (3.1-1) ...
/usr/lib/python3.9/subprocess.py:935: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
  File "/usr/lib/python3/dist-packages/thingsboard_gateway/connectors/opcua_asyncio/device.py", line 23
    if child := re.search(r"(ns=\d*;[isgb]=.*\d)", node_config['path']):
              ^
SyntaxError: invalid syntax

dpkg: error processing package python3-thingsboard-gateway (--configure):
 installed python3-thingsboard-gateway package post-installation script subprocess returned error exit status 1
Processing triggers for man-db (2.8.5-2) ...
Processing triggers for libc-bin (2.28-10+rpt2+rpi1+deb10u1) ...
Errors were encountered while processing:
 python3-thingsboard-gateway
E: Sub-process /usr/bin/dpkg returned an error code (1)```

[samson0v]

Hi @alexcorvis84, this bug was fixed in the master branch. Until the release, you can use Python 3.11.