teslamate icon indicating copy to clipboard operation
teslamate copied to clipboard

Raspberry Pi: clock_gettime(CLOCK_MONOTONIC, _) failed: Operation not permitted (1)

Open KennethEgholm opened this issue 3 years ago • 65 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

What happened?

The main container of teslamate keeps restarting.

Expected Behavior

No response

Steps To Reproduce

No response

Relevant log output

teslamate_1  | clock_gettime(CLOCK_MONOTONIC, _) failed: Operation not permitted (1)
teslamate_1  | Aborted (core dumped)
teslamate_1  | clock_gettime(CLOCK_MONOTONIC, _) failed: Operation not permitted (1)
teslamate_1  | Aborted (core dumped)
teslamate_1  | clock_gettime(CLOCK_MONOTONIC, _) failed: Operation not permitted (1)
teslamate_1  | Aborted (core dumped)
teslamate_1  | clock_gettime(CLOCK_MONOTONIC, _) failed: Operation not permitted (1)
teslamate_1  | Aborted (core dumped)
teslamate_1  | clock_gettime(CLOCK_MONOTONIC, _) failed: Operation not permitted (1)
teslamate_1  | Aborted (core dumped)

Screenshots

No response

Additional data

No response

Type of installation

Docker

Version

v1.25.2

KennethEgholm avatar Jan 12 '22 17:01 KennethEgholm

ke@new-rasp:~/temp $ uname -a Linux new-rasp 5.10.17-v7l+ #1421 SMP Thu May 27 14:00:13 BST 2021 armv7l GNU/Linux ke@new-rasp:~/temp $ docker -v Docker version 20.10.7, build f0df350 ke@new-rasp:~/temp $

KennethEgholm avatar Jan 12 '22 17:01 KennethEgholm

Same here

vanruix avatar Jan 12 '22 17:01 vanruix

Reverting back to 1.25.0 makes the bootloop stop..

KennethEgholm avatar Jan 12 '22 17:01 KennethEgholm

Similar issues getting the container started. Errors differ though, as provided in #2300

teslamate_1  | =ERROR REPORT==== 12-Jan-2022::17:12:06.303755 ===
teslamate_1  | erts_poll_wait() failed: eperm (1)
teslamate_1  | 
teslamate_1  | 
teslamate_1  | =ERROR REPORT==== 12-Jan-2022::17:12:06.303778 ===
teslamate_1  | erts_poll_wait() failed: eperm (1)
teslamate_1  | 
teslamate_1  | 
teslamate_1  | =ERROR REPORT==== 12-Jan-2022::17:12:06.303792 ===
teslamate_1  | erts_poll_wait() failed: eperm (1)
teslamate_1  | 
teslamate_1  | 
teslamate_1  | =ERROR REPORT==== 12-Jan-2022::17:12:06.303805 ===
teslamate_1  | erts_poll_wait() failed: eperm (1)
teslamate_1  | 
teslamate_1  | 
teslamate_1  | =ERROR REPORT==== 12-Jan-2022::17:12:06.303819 ===
teslamate_1  | erts_poll_wait() failed: eperm (1)

More logs:

teslamate_1  | (no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642008600030132}}
teslamate_1  | (no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642008600030145}}
teslamate_1  | (no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642008600030160}}

Have decided to rollback to 1.25 for the time being.

cwanja avatar Jan 12 '22 17:01 cwanja

Reverting back to 1.25.0 makes the bootloop stop..

Could you tell me how you do this ?

vanruix avatar Jan 12 '22 18:01 vanruix

Reverting back to 1.25.0 makes the bootloop stop..

Could you tell me how you do this ?

Change the yml file from

services:
  teslamate:
    image: teslamate/teslamate:latest

To

services:
  teslamate:
    image: teslamate/teslamate:1.25.0

cwanja avatar Jan 12 '22 18:01 cwanja

@leewillis77 posted a soluation here: https://github.com/adriankumpf/teslamate/issues/2300#issuecomment-1010905276

adriankumpf avatar Jan 12 '22 18:01 adriankumpf

I upgraded directly to v1.25.2 and I cannot load teslamate at port 4000. Grafana on 3000 loads to a login window.

pi@teslamate:~ $ docker-compose --version docker-compose version 1.29.2, build unknown pi@teslamate:~ $ docker --version Docker version 20.10.12, build e91ed57 pi@teslamate:~ $ cat /sys/firmware/devicetree/base/model Raspberry Pi 3 Model B Plus Rev 1.3

I believe it actually upgraded to v1.25.2, how do I find the current installed version?

Appears I'm up-to-date:

pi@teslamate:~ $ docker-compose pull Pulling teslamate ... done Pulling database ... done Pulling grafana ... done Pulling mosquitto ... done pi@teslamate:~ $ docker-compose up -d pi_grafana_1 is up-to-date pi_database_1 is up-to-date pi_mosquitto_1 is up-to-date pi_teslamate_1 is up-to-date

petebocken avatar Jan 12 '22 18:01 petebocken

I upgraded directly to v1.25.2 and I cannot load teslamate at port 4000. Grafana on 3000 loads to a login window.

pi@teslamate:~ $ docker-compose --version docker-compose version 1.29.2, build unknown pi@teslamate:~ $ docker --version Docker version 20.10.12, build e91ed57 pi@teslamate:~ $ cat /sys/firmware/devicetree/base/model Raspberry Pi 3 Model B Plus Rev 1.3

I believe it actually upgraded to v1.25.2, how do I find the current installed version?

Appears I'm up-to-date:

pi@teslamate:~ $ docker-compose pull Pulling teslamate ... done Pulling database ... done Pulling grafana ... done Pulling mosquitto ... done pi@teslamate:~ $ docker-compose up -d pi_grafana_1 is up-to-date pi_database_1 is up-to-date pi_mosquitto_1 is up-to-date pi_teslamate_1 is up-to-date

If you did not modify anything in your YML file and the teslamate service says latest then that is 1.25.2. You can also pull the first couple logs of the teslamate container (if you can, mine was flooded with errors) and it will post the version.

cwanja avatar Jan 12 '22 18:01 cwanja

how do I find the current installed version?

@petebocken docker ps -a

leewillis77 avatar Jan 12 '22 18:01 leewillis77

@leewillis77 posted a soluation here: #2300 (comment)

@leewillis77 - do you happen to have a small or quick right up for your fix as linked above? I suspect you are right, I am on v10 and facing errors. Although, it seems inconsistent what errors people are getting.

cwanja avatar Jan 12 '22 18:01 cwanja

docker ps -a

That doesn't appear to show any versions.. It does show latest cause that's what my config is set to. I did first try switching that to 1.25.0 and try to downgrade, but that didn't appear to downgrade at all.

pi@teslamate:~ $ docker ps -a CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 03668f36cb36 teslamate/teslamate:latest "tini -- /bin/sh /en…" 15 minutes ago Up 47 seconds 0.0.0.0:4000->4000/tcp, :::4000->4000/tcp pi_teslamate_1 a19c57b8f61d teslamate/grafana:latest "/run.sh" 16 minutes ago Up 13 minutes 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp pi_grafana_1 1de50c51422c postgres:12 "docker-entrypoint.s…" 16 minutes ago Up 13 minutes 5432/tcp pi_database_1 bbe0088d545c eclipse-mosquitto:1.6 "/docker-entrypoint.…" 16 minutes ago Up 13 minutes 0.0.0.0:1883->1883/tcp, :::1883->1883/tcp pi_mosquitto_1 46ee2b91c708 hello-world "/hello" 7 months ago Exited (0) 7 months ago hungry_yalow

petebocken avatar Jan 12 '22 18:01 petebocken

I upgraded directly to v1.25.2 and I cannot load teslamate at port 4000. Grafana on 3000 loads to a login window. pi@teslamate:~ $ docker-compose --version docker-compose version 1.29.2, build unknown pi@teslamate:~ $ docker --version Docker version 20.10.12, build e91ed57 pi@teslamate:~ $ cat /sys/firmware/devicetree/base/model Raspberry Pi 3 Model B Plus Rev 1.3 I believe it actually upgraded to v1.25.2, how do I find the current installed version? Appears I'm up-to-date: pi@teslamate:~ $ docker-compose pull Pulling teslamate ... done Pulling database ... done Pulling grafana ... done Pulling mosquitto ... done pi@teslamate:~ $ docker-compose up -d pi_grafana_1 is up-to-date pi_database_1 is up-to-date pi_mosquitto_1 is up-to-date pi_teslamate_1 is up-to-date

If you did not modify anything in your YML file and the teslamate service says latest then that is 1.25.2. You can also pull the first couple logs of the teslamate container (if you can, mine was flooded with errors) and it will post the version.

Yup, mine is throwing errors like crazy:

=ERROR REPORT==== 12-Jan-2022::18:13:49.931687 === erts_poll_wait() failed: eperm (1)

petebocken avatar Jan 12 '22 18:01 petebocken

@cwanja There's not much to say really - download package, and install. I'm not really sure I want to provide step-by-step instructions as I'm not wholly convinced that it's a valid fix. From my understanding of how debian manage their repositories it probably is, and I'm comfortable applying that fix to my server, but I'm not (personally) comfortable declaring it as the "proper" fix for everyone.

leewillis77 avatar Jan 12 '22 18:01 leewillis77

03668f36cb36 teslamate/teslamate:latest "tini -- /bin/sh /en…" 15 minutes ago Up 47 seconds 0.0.0.0:4000->4000/tcp, :::4000->4000/tcp pi_teslamate_1

That tells us that you are running the "latest" version. Which would be 1.25.2. For example, here is mine running 1.25.0 specifically.

pi@teslapi:~/Documents/teslamate $ docker container ls
CONTAINER ID   IMAGE                        COMMAND                  CREATED          STATUS          PORTS                                       NAMES
b5a21097be3d   teslamate/teslamate:1.25.0

cwanja avatar Jan 12 '22 18:01 cwanja

03668f36cb36 teslamate/teslamate:latest "tini -- /bin/sh /en…" 15 minutes ago Up 47 seconds 0.0.0.0:4000->4000/tcp, :::4000->4000/tcp pi_teslamate_1

That tells us that you are running the "latest" version. Which would be 1.25.2.

Figured. But i'm still dead in the water. I went straight from whatever the last version was before today to 1.25.2 and it won't let me load teslamate in the browser.

petebocken avatar Jan 12 '22 18:01 petebocken

03668f36cb36 teslamate/teslamate:latest "tini -- /bin/sh /en…" 15 minutes ago Up 47 seconds 0.0.0.0:4000->4000/tcp, :::4000->4000/tcp pi_teslamate_1

That tells us that you are running the "latest" version. Which would be 1.25.2.

Figured. But i'm still dead in the water. I went straight from whatever the last version was before today to 1.25.2 and it won't let me load teslamate in the browser.

You definitely should be able to downgrade. I have gone from 1.25.0 to .2 back to .0 then to .1 and back to .0. You might need to do a docker system prune in between there to remove any lingering images.

cwanja avatar Jan 12 '22 18:01 cwanja

@petebocken see https://github.com/adriankumpf/teslamate/issues/2300#issuecomment-1010717150 for info on how to downgrade the teslamate image to 1.25.0

leewillis77 avatar Jan 12 '22 18:01 leewillis77

@petebocken see #2300 (comment) for info on how to downgrade the teslamate image to 1.25.0

Yup, that's what I did the first time, but I had to run docker system prune first. I got downgraded back to 1.25.0 and then teslamate opened up fine. Then I tried switching config back to latest and updated and then teslamate won't open again and logs are flooded.

I downgraded to 1.25.0 again and it's working. Soooo apparently this isn't fixed in 1.25.2 or am I dealing with a different issue?

(no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642012829015831}}
(no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642012829015844}}
(no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642012829015863}}
=ERROR REPORT==== 12-Jan-2022::18:40:29.015876 ===
erts_poll_wait() failed: eperm (1)


=ERROR REPORT==== 12-Jan-2022::18:40:29.015889 ===
erts_poll_wait() failed: eperm (1)


=ERROR REPORT==== 12-Jan-2022::18:40:29.015902 ===
erts_poll_wait() failed: eperm (1)

petebocken avatar Jan 12 '22 18:01 petebocken

Ah, teslamate loads fine on 1.25.0, but grafana still sits at a login window.

petebocken avatar Jan 12 '22 18:01 petebocken

Ah, teslamate loads fine on 1.25.0, but grafana still sits at a login window.

admin | admin unless you changed it.

cwanja avatar Jan 12 '22 18:01 cwanja

@petebocken see #2300 (comment) for info on how to downgrade the teslamate image to 1.25.0

Yup, that's what I did the first time, but I had to run docker system prune first. I got downgraded back to 1.25.0 and then teslamate opened up fine. Then I tried switching config back to latest and updated and then teslamate won't open again and logs are flooded.

I downgraded to 1.25.0 again and it's working. Soooo apparently this isn't fixed in 1.25.2 or am I dealing with a different issue?

(no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642012829015831}}
(no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642012829015844}}
(no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642012829015863}}
=ERROR REPORT==== 12-Jan-2022::18:40:29.015876 ===
erts_poll_wait() failed: eperm (1)


=ERROR REPORT==== 12-Jan-2022::18:40:29.015889 ===
erts_poll_wait() failed: eperm (1)


=ERROR REPORT==== 12-Jan-2022::18:40:29.015902 ===
erts_poll_wait() failed: eperm (1)

You have the same errors I do in 1.25.2 which leads us back to @leewillis77 post here. Yet to sit down and try and fix it, but I assume you would be running Debian v10 and that is why you are facing the errors on 1.25.2.

cwanja avatar Jan 12 '22 18:01 cwanja

Ah, teslamate loads fine on 1.25.0, but grafana still sits at a login window.

admin | admin unless you changed it.

Thanks, that worked

petebocken avatar Jan 12 '22 18:01 petebocken

@petebocken see #2300 (comment) for info on how to downgrade the teslamate image to 1.25.0

Yup, that's what I did the first time, but I had to run docker system prune first. I got downgraded back to 1.25.0 and then teslamate opened up fine. Then I tried switching config back to latest and updated and then teslamate won't open again and logs are flooded. I downgraded to 1.25.0 again and it's working. Soooo apparently this isn't fixed in 1.25.2 or am I dealing with a different issue?

(no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642012829015831}}
(no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642012829015844}}
(no logger present) unexpected logger message: {log,error,"~s~n",["erts_poll_wait() failed: eperm (1)\n"],#{error_logger=>#{emulator=>true,tag=>error},time=>1642012829015863}}
=ERROR REPORT==== 12-Jan-2022::18:40:29.015876 ===
erts_poll_wait() failed: eperm (1)


=ERROR REPORT==== 12-Jan-2022::18:40:29.015889 ===
erts_poll_wait() failed: eperm (1)


=ERROR REPORT==== 12-Jan-2022::18:40:29.015902 ===
erts_poll_wait() failed: eperm (1)

You have the same errors I do in 1.25.2 which leads us back to @leewillis77 post here. Yet to sit down and try and fix it, but I assume you would be running Debian v10 and that is why you are facing the errors on 1.25.2.

Yup, I'm on Debian Buster 10. I'll sit tight on 1.25.0. :-)

petebocken avatar Jan 12 '22 18:01 petebocken

So I do not want to solicit this as an "official" fix, but I find this fix working and comfortable applying. https://blog.samcater.com/fix-workaround-rpi4-docker-libseccomp2-docker-20/

Specifically this section:

The steps for libseccomp2 are well documented, as this has been a problem on multiple platforms (not just RPI4). You could do a 'oneshot' installation of a newer version, which can be found here https://github.com/itzg/docker-minecraft-server/issues/755#issuecomment-781615497

Personally I feel the better method is to install it from the Buster Backports repo, which is very safe to add. It also means any future updates to libseccomp will be applied to the Pi.

# Get signing keys to verify the new packages, otherwise they will not install
rpi ~$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 04EE7237B7D453EC 648ACFD622F3D138
# Add the Buster backport repository to apt sources.list
rpi ~$ echo 'deb http://httpredir.debian.org/debian buster-backports main contrib non-free' | sudo tee -a /etc/apt/sources.list.d/debian-backports.list
rpi ~$ sudo apt update
rpi ~$ sudo apt install libseccomp2 -t buster-backports

I was able to follow that and got 1.25.2 working.

pi@teslapi:~/Documents/teslamate $ docker-compose logs teslamate
Attaching to teslamate_teslamate_1
teslamate_1  | 2022-01-12 19:02:54.294 [info] Migrations already up
teslamate_1  | 2022-01-12 19:03:00.696 [info] System Info: Erlang/OTP 24 (emu)
teslamate_1  | 2022-01-12 19:03:00.697 [info] Version: 1.25.2

This will also ensure that any future libseccomp updates are included when I update the Pi.

Apply at your own will and risk.

cwanja avatar Jan 12 '22 19:01 cwanja

I have the same Issue on a Raspberry Pi 3 Model B Rev 1.2, 1.25.0 works fine 1.25.1 Only Grafana 1.25.2 Not starting

Just did the same as cwanja and it worked for me. https://github.com/adriankumpf/teslamate/issues/2302#issuecomment-1011362938

ploegy avatar Jan 12 '22 19:01 ploegy

I have the same Issue on a Raspberry Pi 3 Model B Rev 1.2, 1.25.0 works fine 1.25.1 Only Grafana 1.25.2 Not starting

1.25.1 introduced an error, referenced in #2300, preventing TeslaMate web interface to start. 1.25.2 fixed that error, but seemed to introduce a new range of errors as outlined in this thread. There is a fix back in @leewillis77 comment on #2300 or outlined in my comment above.

Edit: to be clear, the errors in 1.25.2 are not related to TeslaMate specifically. Seems there is a core Linux service (file? Dameon?) that was not updated in v10 for Raspbian OS. Someone can better explain it than I can, I am sure.

cwanja avatar Jan 12 '22 19:01 cwanja

The TeslaMate Docker image is based on the official elixir image which in turn is based on the official erlang image. A month ago, the latter was upgraded from Debian Buster to Bullseye even though the major version did not change: https://github.com/erlang/docker-erlang-otp/commit/f14a791cec08903d5ac34a562861bae675cf2ff8#diff-d12fb60cfa187f3314e96974fb7d9c0bc5f5667838c6d86211ae00cb1d64ccccL1

TeslaMate images built after this date are therefore implicitly also based on Debian Bullseye, although no changes have been made by TeslaMate. Thus, unfortunately, we cannot undo this.

Raspberry Pi users therefore have no choice but to either upgrade Raspbian or perform the workarounds described above.

adriankumpf avatar Jan 12 '22 19:01 adriankumpf

Normally the packages on the host system shouldn't matter when running docker packages. Looks like this is one of the exceptions. I suspect the package is altering the config on the kernel which is shared.

brianmay avatar Jan 12 '22 20:01 brianmay

sudo apt install libseccomp2 -t buster-backports

1.25.2 is working well so far.

dkny1123a avatar Jan 13 '22 01:01 dkny1123a