"Operation not permitted" messages logged after initialization

[zim68]

I want to establish an MPTCP connection between two servers with multiple subflows on each side.

Configuration of both servers

• multiple CPUs
• 4-port 1 Gbps Ethernet interface
• Linux Fedora 38 Server (up-to-date)
• Linux Kernel 6.6.2-101.fc38.x86_64
• mptcpd-0.11-2.fc38 installed
• sudo ip mptcp limit show add_addr_accepted 3 subflows 3

Configuration of server 1

• sudo ip mptcp endpoint show 192.168.3.191 id 1 signal subflow dev eno1 192.168.3.192 id 2 signal subflow dev eno2 192.168.3.193 id 3 signal subflow dev eno3 192.168.3.194 id 4 signal subflow dev eno4

Configuration of server 2

• sudo ip mptcp endpoint show 192.168.3.161 id 1 signal subflow dev eno0 192.168.3.162 id 2 signal subflow dev eno1 192.168.3.163 id 3 signal subflow dev eno2 192.168.3.164 id 4 signal subflow dev eno3

On both servers the mptcpd service could be started but logs "Operation not permitted" messages.

Service logs (reversed order) - mptcpd

12:11 PM add_addr: Operation not permitted 12:11 PM set_limits: Operation not permitted 12:11 PM add_addr: Operation not permitted 12:11 PM set_limits: Operation not permitted 12:11 PM add_addr: Operation not permitted 12:11 PM set_limits: Operation not permitted 12:11 PM add_addr: Operation not permitted 12:11 PM set_limits: Operation not permitted (2 occurrences) 12:11 PM MPTCP single-subflow-per-interface path manager initialized. 12:11 PM MPTCP address advertiser path manager initialized. 12:11 PM Started mptcp.service - Multipath TCP service.

Log message details

set_limits: Operation not permitted CODE_FILE commands.c CODE_FUNC mptcpd_check_genl_error CODE_LINE 71 PRIORITY 3 _BOOT_ID 271ac867969843c09c09ed3c84cfc958 _CAP_EFFECTIVE 1000 _CMDLINE /usr/libexec/mptcpd --log=journal _COMM mptcpd _EXE /usr/libexec/mptcpd _GID 62966 _HOSTNAME HP-DL380-1 _MACHINE_ID fbfba620f3184e6fb75ce66a4090bfe8 _PID 1784 _RUNTIME_SCOPE system _SELINUX_CONTEXT system_u:system_r:mptcpd_t:s0 _SOURCE_REALTIME_TIMESTAMP 1701343190958054 _SYSTEMD_CGROUP /system.slice/mptcp.service _SYSTEMD_INVOCATION_ID 9d552c1842244fda9d963265648ca149 _SYSTEMD_SLICE system.slice _SYSTEMD_UNIT mptcp.service _TRANSPORT journal _UID 62966 __CURSOR s=c3ce83a68e9e4cc99c5dd7f4f91e3a23;i=77b46;b=271ac867969843c09c09ed3c84cfc958;m=17a557b;t=60b5cd489c412;x=80846d23a838ac4f __MONOTONIC_TIMESTAMP 24794491 __REALTIME_TIMESTAMP 1701343190959122

add_addr: Operation not permitted CODE_FILE commands.c CODE_FUNC mptcpd_check_genl_error CODE_LINE 71 PRIORITY 3 _BOOT_ID 271ac867969843c09c09ed3c84cfc958 _CAP_EFFECTIVE 1000 _CMDLINE /usr/libexec/mptcpd --log=journal _COMM mptcpd _EXE /usr/libexec/mptcpd _GID 62966 _HOSTNAME HP-DL380-1 _MACHINE_ID fbfba620f3184e6fb75ce66a4090bfe8 _PID 1784 _RUNTIME_SCOPE system _SELINUX_CONTEXT system_u:system_r:mptcpd_t:s0 _SOURCE_REALTIME_TIMESTAMP 1701343199804620 _SYSTEMD_CGROUP /system.slice/mptcp.service _SYSTEMD_INVOCATION_ID 9d552c1842244fda9d963265648ca149 _SYSTEMD_SLICE system.slice _SYSTEMD_UNIT mptcp.service _TRANSPORT journal _UID 62966 __CURSOR s=c3ce83a68e9e4cc99c5dd7f4f91e3a23;i=77c51;b=271ac867969843c09c09ed3c84cfc958;m=201533a;t=60b5cd510c1d1;x=ebc3b2a80e6ec26f __MONOTONIC_TIMESTAMP 33641274 __REALTIME_TIMESTAMP 1701343199805905

Expected behavior

There are no errors/warnings after service mptcpd has been started and initialized.

[zim68]

UPDATE: After deactivation of the plugin "addr_adv" in /etc/mptcp/mptcp.conf and restarting the service "mptcpd" the error messages will not print anymore.

log=stderr plugin-dir=/usr/lib64/mptcpd path-manager=sspi addr-flags=signal,subflow notify-flags=existing,skip_link_local,skip_loopback load-plugins=sspi

But that's not the goal to be achieved. So it is clear that this plugin triggers these error messages.

What did I configure incorrectly?

[ivanbaldo]

Happening to me in an up to date RHEL 9.4 server...

[ossama-othman]

The set_limits and add_addr operations executed by the addr_adv plugin require the CAP_NET_ADMIN capability if not running as root. Otherwise those operations will result in the "Operation not permitted" (EPERM) error that you're seeing.

The mptcpd.service systemd unit file used to start mptcpd should already set up the necessary capabilities for the mptcp dynamic user under which mptcpd runs. If you're running mptcpd without systemd and want to use the addr_adv plugin without running as root, you'll have to explicitly set the CAP_NET_ADMIN capability, as described in the README.

[ivanbaldo]

With the mptcpd-0.8-3.el9 package installed it doesn't work. If I run it as root manually with /usr/libexec/mptcpd then it does work. Maybe some other capability is needed too?

# systemctl cat mptcp
# /usr/lib/systemd/system/mptcp.service
# SPDX-License-Identifier: BSD-3-Clause
#
# Copyright (c) 2017-2019, Intel Corporation

[Unit]
Description=Multipath TCP service
Documentation=man:mptcpd(8)

[Service]
Type=simple
DynamicUser=yes
ExecStart=/usr/libexec/mptcpd --log=journal
CapabilityBoundingSet=CAP_NET_ADMIN
AmbientCapabilities=CAP_NET_ADMIN
LimitNPROC=1

[Install]
WantedBy=sockets.target

Thanks!!