"Unsupported system - Systemd Journal issues" warning since 2022.11.1

[fthiery]

Describe the bug

Since Home Assistant 2022.11.1 / Supervisor 2022.10.2 a warning "Unsupported system - Systemd Journal issues" appears.

To Reproduce

1. Updated all items using the WebUI
2. Updated the hafbx host (apt upgrade)

The warning appears.

Expected behavior

No warning

Screenshots

Additional context

I tried to run apt install systemd-journal-remote -y but it won't start

ha@hafbx:~$ systemctl status systemd-journal-remote
● systemd-journal-remote.service - Journal Remote Sink Service
     Loaded: loaded (/lib/systemd/system/systemd-journal-remote.service; indirect; vendor preset: disabled)
     Active: failed (Result: exit-code) since Fri 2022-11-04 11:54:55 CET; 38min ago
TriggeredBy: ● systemd-journal-remote.socket
       Docs: man:systemd-journal-remote(8)
             man:journal-remote.conf(5)
    Process: 7896 ExecStart=/lib/systemd/systemd-journal-remote --listen-https=-3 --output=/var/log/journal/remote/ (code=exited, status=1/FAILURE)
   Main PID: 7896 (code=exited, status=1/FAILURE)
      Error: 13 (Permission denied)

Nov 04 11:54:55 hafbx systemd[1]: Started Journal Remote Sink Service.
Nov 04 11:54:55 hafbx systemd-journal-remote[7896]: Failed to read key from file '/etc/ssl/private/journal-remote.pem': Permission denied
Nov 04 11:54:55 hafbx systemd[1]: systemd-journal-remote.service: Main process exited, code=exited, status=1/FAILURE
Nov 04 11:54:55 hafbx systemd[1]: systemd-journal-remote.service: Failed with result 'exit-code'.

[Feragorn]

I had the same problem and not solved... BUT I used sudo apt install systemd-journal-remote -y And it installed well... However, the warning is still here

According to the "help page", we should "repair" the Supervisor using the supervised installer... I stopped at this point (it was 2am and I needed to sleep)

[fthiery]

According to the "help page", we should "repair" the Supervisor using the supervised installer... I stopped at this point (it was 2am and I needed to sleep)

I did run dpkg-reconfigure homeassistant-supervised without success

[fthiery]

This seems to be the upstream issue report (or at least related): https://github.com/home-assistant/supervised-installer/issues/247

[fthiery]

I noticed that homeassistant-supervised.deb is only fetched on first boot and not installed by a repo (https://github.com/foreign-sub/ha_fbx/blob/master/nocloud-net/user-data#L87), so it does not get updated (i was still on 1.2.1).

Is there an recommended way to update the ha_fbx VM (other than redeploy a fresh image) ?

I tried this without success:

apt update && apt -y dist-upgrade
apt install -y systemd-journal-remote
wget -q https://github.com/home-assistant/supervised-installer/releases/latest/download/homeassistant-supervised.deb
dpkg -i homeassistant-supervised.deb
rm homeassistant-supervised.deb

systemctl status systemd-journal-remote shows:

Nov 04 11:54:55 hafbx systemd-journal-remote[7896]: Failed to read key from file '/etc/ssl/private/journal-remote.pem': Permission denied

I see nothing in this repo involving the creation of the SSL certificate, i'm assuming that this project is not yet able to support the latest HA version.

[Feragorn]

Wow... you went so farther than I did... I'll try it tonight...

Well, I have to admit that I'm a little surprised regarding the difference between 2 releases : 0. So if everything depends on the initial script, we might find an answer in the "official" script (I'm searching inside the official images) https://github.com/home-assistant/operating-system/releases

I think I update my system like you using ssh and apt-get...

[fthiery]

Apparently nobody cares that systemd-journal-remote does not work, the important bit is that the following does (using systemd-journal-gatewayd.service under the hood):

curl 'http://localhost:19531/entries?boot'

[Feragorn]

https://community.home-assistant.io/t/journal-issues-with-supervised-install-after-updating-to-2022-11/483879/8

https://community.home-assistant.io/t/journal-issues-with-supervised-install-after-updating-to-2022-11/483879/23

[blendrik]

For RaspberryPi OS:

The file /usr/lib/systemd/system/systemd-journal-gatewayd.socket installed by homeassistant-supervised.deb is placed in the wrong directory. On my system it's located in /lib/systemd/system/systemd-journal-gatewayd.socket. So I commented out the line with the port and added the sock-file as so

[Socket]
#ListenStream=19531
ListenStream=/run/systemd-journal-gatewayd.sock

With this, at least I got rid of the systemd-journal-issues and can see system logs within HA.

[fthiery]

I don't understand why the /run/systemd-journal-gatewayd.sock is a folder (and what creates it). I think it's the root issue.

I believe the impact is that host logs do not appear on http://192.168.1.79:8123/config/logs

file /run/systemd-journal-gatewayd.sock
/run/systemd-journal-gatewayd.sock: directory

I tried https://github.com/home-assistant/supervised-installer/issues/247#issuecomment-1302634835 but after the reboot the sock file is back to a folder.

[blendrik]

@fthiery You have to delete the directory /run/systemd-journal-gatewayd.sock. Then follow my advice and change the right systemd-journal-gatewayd.socket-file. The directory is created when docker doesn't find a (sock-)file. I don't why this is the case. But it is created via the /usr/sbin/hassio-supervisor-script:

-v /run/systemd-journal-gatewayd.sock:/run/systemd-journal-gatewayd.sock:rw \

docker container create \
        --name hassio_supervisor \
        --privileged --security-opt apparmor="hassio-supervisor" \
        -v /run/docker.sock:/run/docker.sock:rw \
        -v /run/systemd-journal-gatewayd.sock:/run/systemd-journal-gatewayd.sock:rw \
        -v /run/dbus:/run/dbus:ro \
        -v /run/supervisor:/run/os:rw \
        -v /run/udev:/run/udev:ro \
        -v /etc/machine-id:/etc/machine-id:ro \
        -v ${SUPERVISOR_DATA}:/data:rw \
        -e SUPERVISOR_SHARE=${SUPERVISOR_DATA} \
        -e SUPERVISOR_NAME=hassio_supervisor \
        -e SUPERVISOR_MACHINE=${SUPERVISOR_MACHINE} \
        "${SUPERVISOR_IMAGE}:latest"

[fthiery]

It ends up being the fact that the systemd config file /lib/systemd/system/systemd-journal-gatewayd.socket shipped in the homeassistant-supervised package is configured to work as TCP socket instead of an UNIX socket (which is what home assistant expects on the /run/systemd-journal-gatewayd.sock path for gettting system logs).

Patching the /lib/systemd/system/systemd-journal-gatewayd.socket file to create a UNIX socket with ListenStream=/run/systemd-journal-gatewayd.sock makes systemd create the socket file (instead of allocating the TCP port); the folder was probably created by docker by trying to bindmount to a missing file.

$ file /run/systemd-journal-gatewayd.sock
/run/systemd-journal-gatewayd.sock: socket

I'm guessing that at some point either someone will have to change the socket file provided by homeassistant-supervised, or we will need to create a systemd override file.

Thanks @blendrik

To sum up:

1. update everying using the UI

2. install systemd-journal-remote and update homeassistant-supervised.deb

apt install -y systemd-journal-remote
wget -q https://github.com/home-assistant/supervised-installer/releases/latest/download/homeassistant-supervised.deb
dpkg -i homeassistant-supervised.deb
rm homeassistant-supervised.deb

3. edit /lib/systemd/system/systemd-journal-gatewayd.socket to replace #ListenStream=19531 by ListenStream=/run/systemd-journal-gatewayd.sock

4. reboot

5. enjoy

[fthiery]

The nocloud-net/user-data file will probably require matching modifications

[ioannispelelis]

It ends up being the fact that the systemd config file /lib/systemd/system/systemd-journal-gatewayd.socket shipped in the homeassistant-supervised package is configured to work as TCP socket instead of an UNIX socket (which is what home assistant expects on the /run/systemd-journal-gatewayd.sock path for gettting system logs).

Patching the /lib/systemd/system/systemd-journal-gatewayd.socket file to create a UNIX socket with ListenStream=/run/systemd-journal-gatewayd.sock makes systemd create the socket file (instead of allocating the TCP port); the folder was probably created by docker by trying to bindmount to a missing file.

$ file /run/systemd-journal-gatewayd.sock
/run/systemd-journal-gatewayd.sock: socket

I'm guessing that at some point either someone will have to change the socket file provided by homeassistant-supervised, or we will need to create a systemd override file.

Thanks @blendrik

To sum up:

1. update everying using the UI
2. install systemd-journal-remote and update homeassistant-supervised.deb

apt install -y systemd-journal-remote
wget -q https://github.com/home-assistant/supervised-installer/releases/latest/download/homeassistant-supervised.deb
dpkg -i homeassistant-supervised.deb
rm homeassistant-supervised.deb

3. edit /lib/systemd/system/systemd-journal-gatewayd.socket to replace #ListenStream=19531 by ListenStream=/run/systemd-journal-gatewayd.sock
4. reboot
5. enjoy

looks like your method worked for me as well. However i did not have to edit the file in step 3. the correct line in the file was already present.

[Hamper]

I think that /lib/systemd/system/systemd-journal-gatewayd.socket can be reverted on apt update, in that case is better to create /etc/systemd/system/systemd-journal-gatewayd.socket.d/hass-socket.conf:

[Socket]
ListenStream=/run/systemd-journal-gatewayd.sock