ioBroker.docker icon indicating copy to clipboard operation
ioBroker.docker copied to clipboard

[v6.1] bash: /usr/bin/node: Operation not permitted

Open viceice opened this issue 3 years ago • 15 comments

After upgrading jscontroller from v3.3.x to v4.0.15, i get EPERM on logs for starting instances.

When trying manually starting node from root bash in container i get bash: /usr/bin/node: Operation not permitted

viceice avatar Mar 03 '22 10:03 viceice

I'm using ghcr.io/buanet/iobroker:v6.1.0@sha256:d8c2abe61385cd0a1462f3a8d9a001de33d58df43cc724e59283545a8cb080b6

viceice avatar Mar 03 '22 10:03 viceice

OK, deleting and recreating container (with existing mounts/config) seems to fix the issue 😕

viceice avatar Mar 03 '22 10:03 viceice

Controller 4.0 should normally detect the needed capabilities to set automatically when he updates the capabilties... We made sure this happens correctly ... there was an "in between version" of the 4.0.x beta versions that broke that ...

Apollon77 avatar Mar 03 '22 11:03 Apollon77

I have a similar issue which is reproducable. When I'm starting a v5.2.0 image in my microk8s environment with an empty volume, everthing works as expected. I can install adapters and communicate with devices.

Now I do the same with a v6.1.0 - the one you've pushed 4 hours ago - as well with an empty volume. In this case I can run the welcome/setup wizzard and finish it, but the expected device discovery dialog does not appear. Instead I get this in the logs:

Cannot get NPM version: Error: Command failed: npm -v /usr/bin/env: ‘node’: Operation not permitted

Restarting the host with the GUI does not bring back the web server. After deleting the pod k8s brings up a new container based on the previously created volume. Now I can install adapters and communicate with devices.

So just the first start is a problem.

DizzelPopitzel avatar Mar 14 '22 12:03 DizzelPopitzel

I am running a K3S single node cluster on a raspberry pi 4 and when using the docker image buanet/iobroker:v6.1.0-arm64v8 from docker hub I get the following errors messages:

  • In admin UI logs: Error: spawn EPERM for all adapters except admin
  • In a terminal: /usr/bin/env: ‘node’: Operation not permitted when I try execute npm -v

During container startup process no errors are reported.

The pi is running raspios based on debian bullseye with kernel 5.10.103-v8+ (the most recent from standard apt repos). The volume is a local-path on the pi where I mounted an NFS share via /etc/fstab (noexec is not set).

The already mentioned workaround (deleting and recreating the pod) does not work in my case. Further, I tried iobroker fix script without any success. When I run an older image tag (for instance v5.2.0-arm64v8), iobroker works without errors.

Do you have any idea where I could further check what is wrong when using the most recent docker image?

jarvisquis avatar Mar 17 '22 17:03 jarvisquis

which js-controller ist used?

Apollon77 avatar Mar 17 '22 17:03 Apollon77

js-controller 4.0.21 is used

jarvisquis avatar Mar 17 '22 18:03 jarvisquis

the node capabilities need to be set anew ... so iob fix should be executed

Apollon77 avatar Mar 17 '22 20:03 Apollon77

ok I stopped admin adapter via WebUI and executed from within the container as root iob fix from /opt/iobroker. The output of the script looks fine for me:

root@iobroker-67cf4877f5-przqj:/opt/iobroker# iob fix
library: loaded
Library version=2022-02-10

==========================================================================

    Welcome to the ioBroker installation fixer!
    Script version: 2022-02-13

==========================================================================


==========================================================================
    Installing prerequisites (1/3)
==========================================================================

Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://security.debian.org/debian-security bullseye-security InRelease [44,1 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [39,4 kB]        
Get:4 https://deb.nodesource.com/node_14.x bullseye InRelease [4.586 B]        
Get:5 http://deb.debian.org/debian bullseye/main arm64 Packages [8.070 kB]
Get:6 http://deb.debian.org/debian bullseye-updates/main arm64 Packages [2.596 B]
Get:7 http://security.debian.org/debian-security bullseye-security/main arm64 Packages [121 kB]
Get:8 https://deb.nodesource.com/node_14.x bullseye/main arm64 Packages [773 B]
Fetched 8.398 kB in 4s (2.271 kB/s)
Reading package lists... Done
Installed gcc-c++
Installed python-dev

==========================================================================
    Checking ioBroker user and directory permissions (2/3)
==========================================================================

Created /etc/sudoers.d/iobroker
Fixing directory permissions...
This system does not support setting default permissions.
Do not use npm to manually install adapters unless you know what you are doing!

==========================================================================
    Checking autostart (3/3)
==========================================================================

Unsupported init system, cannot enable autostart!

==========================================================================

    Your installation was fixed successfully
    Run iobroker start to start ioBroker again!

==========================================================================

Then i tried to execute iob commands. But again the following errors occured:

root@iobroker-67cf4877f5-przqj:/opt/iobroker# iob start
sudo: unable to execute /usr/bin/node: Operation not permitted
root@iobroker-67cf4877f5-przqj:/opt/iobroker# iob
sudo: unable to execute /usr/bin/node: Operation not permitted

jarvisquis avatar Mar 18 '22 15:03 jarvisquis

What is the result of getcap $(eval readlink -f `which node`)??

@DizzelPopitzel Do you use a raspberry pi 4 with arm64v8 image too??

buanet avatar Mar 18 '22 17:03 buanet

Yes, raspberry pi 4 with arm64v8 image

DizzelPopitzel avatar Mar 18 '22 19:03 DizzelPopitzel

Output of getcap is:

/usr/bin/node cap_net_bind_service,cap_net_admin,cap_net_raw=eip

Further, when I delete the capability with setcap -r, npm -v command does work. When I delete the pod afterwards the pod gets automatically restarted and then I can use iobroker without errors.

jarvisquis avatar Mar 18 '22 20:03 jarvisquis

Just for note, running the docker image in a kubernetes cluster is not supported so far, as I did not tested it. So it might work, but you also can face issues like this one :)

BUT

After some testing, I think I figured out the root cause. As @Apollon77 said your capabilities are not set correctly. As long as cap_net_admin is set on node, but not set for the container you will get the "Permission denied" issue. So there are two ways to hopefully get your setup working:

  1. Set cap_net_admin to your container (docker docs link)
  2. Correct the capabilities (removing cap_net_admin) inside your container with
sudo setcap 'cap_net_bind_service,cap_net_raw=eip' `which node`

Can you try if this workaround works?

@Apollon77 @AlCalzone I think we have to talk about our capabilities check in js-controller to detect kubernetes as we detect docker. As far as I could determine this works for docker but not for kubernetes.

Regards, André

buanet avatar Mar 18 '22 23:03 buanet

Great! Adding cap_net_admin makes it run as expected. Thanks for your support.

DizzelPopitzel avatar Mar 19 '22 09:03 DizzelPopitzel

@buanet Yes I know that this is not officially supported and I appreciate the support here ;)

Both proposed workarounds fix the issue. Thank You

jarvisquis avatar Mar 20 '22 19:03 jarvisquis

We did some tweaks to the Docker detection of ioBroker. Hopefully this prevents us from wrong capabilities settings as docker is detected more reliable. However, I'll close this at this point. Thank you.

Regards, André

buanet avatar Dec 07 '22 10:12 buanet