Apama software management plugin

[NATSIRTIOT]

Describe the bug When following the guide to install apama artefacts on a device (https://github.com/thin-edge/thin-edge.io/blob/main/docs/src/howto-guides/017_apama_software_management_plugin.md), I got a 401 Unauthorized error.

To Reproduce

• Thin-edge.io and apama are already installed on my device (Raspberry Pi4B, Raspberry Pi OS Lite 32-bit).

• Running apama projects works as well.

• I followed these steps:

curl -LJO https://github.com/thin-edge/thin-edge.io/releases/download/0.5.0/tedge_apama_plugin_0.6.0_armhf.deb
sudo dpkg -i tedge_apama_plugin_0.6.0_armhf.deb

• Added https://github.com/thin-edge/thin-edge.io/raw/main/tests/PySys/plugin_apama/Input/quickstart.zip to the Software repository in Cumolocity IoT:

• After Install software > Apply changes I got the following error:

The mentioned log file in the error is added below.

Expected behavior Install apama projects on my Raspberry Pi using Cumulocity software management feature.

Log file /var/log/tedge/agent/software-update-2022-04-19T11:14:27.928531927Z.log

----- $ sudo "/etc/tedge/sm-plugins/apama" "prepare"
exit status: 0

stdout <<EOF
EOF

stderr <<EOF
EOF
----- $ Downloading: https://env561183.eu-latest.cumulocity.com/inventory/binaries/10201 to /tmp/co2-project::project_2.0
error: DownloadError error: "JSON parse error: \"HTTP status client error (401 Unauthorized) for url (https://env561183.eu-latest.cumulocity.com/inventory/binaries/10201)\"" for "https://env561183.eu-latest.cumulocity.com/inventory/binaries/10201"
----- $ sudo "/etc/tedge/sm-plugins/apama" "finalize"
exit status: 0

stdout <<EOF
EOF

stderr <<EOF
EOF
----- $ sudo "/etc/tedge/sm-plugins/apama" "list"
exit status: 0

stdout <<EOF
CO2::project
SensorMonitor::mon
EOF

stderr <<EOF
EOF
----- $ sudo "/etc/tedge/sm-plugins/apt" "list"
exit status: 0

stdout <<EOF
adduser 3.118
apt-listchanges 3.24
apt-utils       2.2.4
apt     2.2.4
avahi-daemon    0.8-5
base-files      11.1+rpi1+deb11u3
base-passwd     3.5.51
bash-completion 1:2.11-2
bash    5.1-2
bridge-utils    1.7-1
bsdextrautils   2.36.1-8+deb11u1
bsdutils        1:2.36.1-8+deb11u1
build-essential 12.9
ca-certificates 20210119
cifs-utils      2:6.11-3.1
co2debpkg       1.0
console-setup   1.205
coreutils       8.32-4
cpio    2.13+dfsg-4
crda    4.14+git20191112.9856751-1
cron    3.0pl1-137
curl    7.74.0-1.3+deb11u1
dash    0.5.11+git20200708+dd9ef66-5
debconf-i18n    1.5.77
debconf-utils   1.5.77
debconf 1.5.77
debianutils     4.11.2
dhcpcd5 1:8.1.2-1+rpt5
diffutils       1:3.7-5
dirmngr 2.2.27-2+deb11u1
dmidecode       3.3-2
dmsetup 2:1.02.175-2.1
dnsmasq 2.85-1
dosfstools      4.2-1
dphys-swapfile  20100506-7
dpkg    1.20.9+rpi1
e2fsprogs       1.46.2-2
ed      1.17-1
ethtool 1:5.9-1
fake-hwclock    0.12
fbset   2.1-32
fdisk   2.36.1-8+deb11u1
findutils       4.8.0-1
firmware-atheros        1:20210315-3+rpt5
firmware-brcm80211      1:20210315-3+rpt5
firmware-libertas       1:20210315-3+rpt5
firmware-misc-nonfree   1:20210315-3+rpt5
firmware-realtek        1:20210315-3+rpt5
gcc-10-base     10.2.1-6+rpi1
gcc-7-base      7.5.0-6+rpi1+b2
gcc-8-base      8.4.0-7+rpi1
gcc-9-base      9.3.0-22+rpi1
gdb     10.1-1.7
git     1:2.30.2-1
gnupg-l10n      2.2.27-2+deb11u1
gnupg-utils     2.2.27-2+deb11u1
gnupg   2.2.27-2+deb11u1
gpg-agent       2.2.27-2+deb11u1
gpg-wks-client  2.2.27-2+deb11u1
gpg-wks-server  2.2.27-2+deb11u1
gpg     2.2.27-2+deb11u1
gpgconf 2.2.27-2+deb11u1
gpgsm   2.2.27-2+deb11u1
gpgv    2.2.27-2+deb11u1
grep    3.6-1
groff-base      1.22.4-6
gzip    1.10-4
hardlink        0.3.2+b1
hostapd 2:2.9.0-21
hostname        3.23
htop    3.0.5-7
ifupdown        0.8.36
init-system-helpers     1.60
init    1.60
install-info    6.7.0.dfsg.2-6
iproute2        5.10.0-4
iptables-persistent     1.0.15
iptables        1.8.7-1
iputils-ping    3:20210202-1
isc-dhcp-client 4.4.1-2.3
isc-dhcp-common 4.4.1-2.3
keyboard-configuration  1.205
kmod    28-1
less    551-2
libacl1 2.2.53-10
libapparmor1    2.13.6-10
libapt-pkg6.0   2.2.4
libargon2-1     0~20171227-0.2
libassuan0      2.5.3-7.1
libatomic1      10.2.1-6+rpi1
libattr1        1:2.4.48-6
libaudit-common 1:3.0-2
libaudit1       1:3.0-2
libblkid1       2.36.1-8+deb11u1
libbpf0 1:0.3-2+rpi1
libbsd0 0.11.3-1
libbz2-1.0      1.0.8-4
libc-bin        2.31-13+rpt2+rpi1+deb11u2
libc6   2.31-13+rpt2+rpi1+deb11u2
libcamera-apps-lite     0~git20220228+0dc5ea0-1
libcap-ng0      0.7.9-2.2+b1
libcap2-bin     1:2.44-1
libcap2 1:2.44-1
libcom-err2     1.46.2-2
libcrypt1       1:4.4.18-4
libcryptsetup12 2:2.3.7-1+deb11u1
libdb5.3        5.3.28+dfsg1-0.8
libdebconfclient0       0.260
libdevmapper1.02.1      2:1.02.175-2.1
libdns-export1110       1:9.11.19+dfsg-2.1
libedit2        3.1-20191231-2
libelf1 0.183-1
libestr0        0.1.10-2.1
libext2fs2      1.46.2-2
libfastjson4    0.99.9-1
libfdisk1       2.36.1-8+deb11u1
libffi7 3.3-6
libfreetype6-dev        2.10.4+dfsg-1
libgcc-s1       10.2.1-6+rpi1
libgcrypt20     1.8.7-6
libgdbm6        1.19-2
libgmp10        2:6.2.1+dfsg-1+deb11u1
libgnutls30     3.7.1-5
libgpg-error0   1.38-2
libgssapi-krb5-2        1.18.3-6+deb11u1
libhogweed6     3.7.3-1
libident        0.22-3.1
libidn2-0       2.3.0-5
libip4tc2       1.8.7-1
libisc-export1105       1:9.11.19+dfsg-2.1
libjansson4     2.13.1-1.1
libjson-c5      0.15-2
libk5crypto3    1.18.3-6+deb11u1
libkeyutils1    1.6.1-2
libkmod2        28-1
libkrb5-3       1.18.3-6+deb11u1
libkrb5support0 1.18.3-6+deb11u1
libksba8        1.5.0-3
libldap-2.4-2   2.4.57+dfsg-3
liblocale-gettext-perl  1.07-4+b1
liblognorm5     2.0.5-1.1
liblz4-1        1.9.3-2
liblzma5        5.2.5-2
libmd0  1.0.3-3
libmnl0 1.0.4-3
libmount1       2.36.1-8+deb11u1
libmtp-runtime  1.1.17-3
libncurses6     6.2+20201114-2
libncursesw6    6.2+20201114-2
libnettle8      3.7.3-1
libnftables1    0.9.8-3.1
libnftnl11      1.1.9-1
libnpth0        1.6-3
libnsl2 1.3.0-2
libp11-kit0     0.23.22-1
libpam-chksshpwd        1.4.0-9+deb11u1+rpt1
libpam-modules-bin      1.4.0-9+deb11u1+rpt1
libpam-modules  1.4.0-9+deb11u1+rpt1
libpam-runtime  1.4.0-9+deb11u1+rpt1
libpam0g        1.4.0-9+deb11u1+rpt1
libpcre2-8-0    10.36-2
libpcre3        2:8.39-13
libpipeline1    1.5.3-1
libpopt0        1.18-2
libprocps8      2:3.3.17-5
libraspberrypi-bin      1:2+git20211125~155417+14b90ff-3
libraspberrypi-dev      1:2+git20211125~155417+14b90ff-3
libraspberrypi-doc      1:2+git20211125~155417+14b90ff-3
libraspberrypi0 1:2+git20211125~155417+14b90ff-3
libreadline6    6.3-9
libreadline8    8.1-1
libsasl2-2      2.1.27+dfsg-2.1+deb11u1
libsasl2-modules-db     2.1.27+dfsg-2.1+deb11u1
libseccomp2     2.5.1-1+rpi1+deb11u1
libselinux1     3.1-3
libsemanage-common      3.1-1
libsemanage1    3.1-1+b1
libsepol1       3.1-1
libsigc++-1.2-5c2       1.2.7-2+b1
libsmartcols1   2.36.1-8+deb11u1
libsqlite3-0    3.34.1-3
libss2  1.46.2-2
libssl1.1       1.1.1n-0+deb11u1
libstdc++6      10.2.1-6+rpi1
libsystemd0     247.3-7+rpi1
libtasn1-6      4.16.0-2
libtext-charwidth-perl  0.04-10+b1
libtext-iconv-perl      1.7-7+b1
libtext-wrapi18n-perl   0.06-9
libtinfo5       6.2+20201114-2
libtinfo6       6.2+20201114-2
libtirpc-common 1.3.1-1
libtirpc3       1.3.1-1
libuchardet0    0.0.7-1
libudev1        247.3-7+rpi1
libunistring2   0.9.10-4
libuuid1        2.36.1-8+deb11u1
libxtables12    1.8.7-1
libxxhash0      0.8.0-2+rpi1
libzstd1        1.4.8+dfsg-2.1+rpi1
locales 2.31-13+rpt2+rpi1+deb11u2
login   1:4.8.1-1
logrotate       3.18.0-2
logsave 1.46.2-2
lsb-base        11.1.0+rpi1
lua5.1  5.1.5-8.1+b2
luajit  2.1.0~beta3+dfsg-5.3
man-db  2.9.4-2
manpages-dev    5.10-1
mawk    1.3.4.20200120-2
mkvtoolnix      54.0.0+really52.0.0-3
mosquitto       2.0.11-1
mount   2.36.1-8+deb11u1
nano    5.4-2
ncdu    1.15.1-1
ncurses-base    6.2+20201114-2
ncurses-bin     6.2+20201114-2
net-tools       1.60+git20181103.0eebece-1
netbase 6.3
netcat-openbsd  1.217-3
nfs-common      1:1.3.4-6
nftables        0.9.8-3.1
ntfs-3g 1:2017.3.23AR.3-4+deb11u1
openssl 1.1.1n-0+deb11u1
p7zip-full      16.02+dfsg-8
parted  3.4-1
passwd  1:4.8.1-1
paxctld 1.2.1-1
pciutils        1:3.7.0-5
perl-base       5.32.1-4+deb11u2
pi-bluetooth    0.1.18
pinentry-curses 1.1.0-4
pkg-config      0.29.2-1
policykit-1     0.105-31+rpt1+deb11u1
procps  2:3.3.17-5
psmisc  23.4-2
python-is-python3       3.9.2-1
python3-gpiozero        1.6.2-1
python3-pip     20.3.4-4+rpt1
python3-rpi.gpio        0.7.0-0.2+b1
python3 3.9.2-3
raspberrypi-archive-keyring     2021.1.1+rpt1
raspberrypi-bootloader  1:1.20220331-1
raspberrypi-kernel      1:1.20220331-1
raspberrypi-net-mods    1.3.3
raspberrypi-sys-mods    20220224
raspbian-archive-keyring        20120528.2
raspi-config    20220331
raspi-copies-and-fills  0.13
raspinfo        20190624-1
readline-common 8.1-1
rng-tools       5migrate1
rpi-eeprom      13.12-1
rpi-update      20210618
rsync   3.2.3-4+deb11u1
rsyslog 8.2102.0-2
sed     4.7-1
sensible-utils  0.0.14
ssh-import-id   5.10-1
ssh     1:8.4p1-5
strace  5.10-1
sudo    1.9.5p2-3
systemd-sysv    247.3-7+rpi1
systemd-timesyncd       247.3-7+rpi1
systemd 247.3-7+rpi1
sysvinit-utils  2.96-7+deb11u1
tar     1.34+dfsg-1
tasksel-data    3.68+deb11u1
tasksel 3.68+deb11u1
tedge   0.6.1
tedge_agent     0.6.0
tedge_apama_plugin      0.6.0
tedge_apt_plugin        0.6.0
tedge_logfile_request_plugin    0.6.1
tedge_mapper    0.6.1
traceroute      1:2.1.0-2
tzdata  2021a-1+deb11u2
udev    247.3-7+rpi1
udisks2 2.9.2-2+deb11u1
unzip   6.0-26
usb-modeswitch  2.6.1-1
usbutils        1:013-3
util-linux      2.36.1-8+deb11u1
v4l-utils       1.20.0-2
vim-common      2:8.2.2434-3+deb11u1
vim-tiny        2:8.2.2434-3+deb11u1
wireless-tools  30~pre9-13.1
wpasupplicant   2:2.9.0-21
xxd     2:8.2.2434-3+deb11u1
zip     3.0-12
zlib1g  1:1.2.11.dfsg-2+deb11u1
EOF

stderr <<EOF

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

EOF

Environment (please complete the following information):

• Raspberry Pi OS Lite 32-bit
• Raspberry Pi 4B
• armhf
• thin-edge.io version 0.6.0

Additional context: Running my custom apama projects works. But then I need to copy the project folder from my laptop to the Raspberry Pi manually.

[didier-wenzek]

Thank you for this detailed issue.

This bug seems to be due to a timing issues unrelated to Apama: the JWT token used to download the Apama project expires before being used. Faced to such an unauthorized error, the mapper should renew its JWT token instead of giving up. Can you eventually install the Apama project from Cumulocity if you manually retry?

We will work on this issue.

[NATSIRTIOT]

Thank you for your fast answer.

I'm glad to hear that you will work on this issue. What do you exactly mean by manually retrying?

[didier-wenzek]

I mean by manually retrying to retry the update from Cumulocity. I suspect a timing issue, but want to be sure.

[reubenmiller]

Since the retry of the jwt token was merged I suspect this will fix the issue #1586