FDio
[VPP-1679] SElinux rules are not set accordingly
Description
After compiling VPP 19.04 with support for MLX5 DPDK driver and installing it, VPP is unable to access the DPDK devices. Neither is it able to create a new RDMA device. This happens on CentOS 7.6 and RHEL 7.6. The problem exists in 18.04, 18.10 and 19.01 as well but in these versions the RDMA part is not relevant.
vpp# create int rdma host-if enp1s0f1 name rdma-0 create interface rdma: no RDMA devices available, errno = 13. Is the ib_uverbs module loaded?: Permission denied
vpp# show pci Address Sock VID:PID Link Speed Driver Product Name Vital Product Data0000:01:00.0 15b3:1013 8.0 GT/s x8 mlx5_core CX414A - ConnectX-4 QSFP28 PN: MCX414A-BCAT EC: A7 SN: MT1622X00459 V0: 0x 50 43 49 65 47 65 6e 33 ... RV: 0x 400000:01:00.1 15b3:1013 8.0 GT/s x8 mlx5_core CX414A - ConnectX-4 QSFP28 PN: MCX414A-BCAT EC: A7
sudo setenforce 0
Version used:
vpp# show version verbose cmdline
Version: v19.04.1-rc0~6-g725c6c4
Compiled by: centos
Compile host: node6
Compile date: Wed May 15 10:03:13 UTC 2019
Compile location: /home/centos/vpp.1904
Compiler: GCC 7.3.1 20180303 (Red Hat 7.3.1-5)
Current PID: 30798
Command line arguments:
/usr/bin/vpp
unix
{ nodaemon log /var/log/vpp/vpp.log full-coredump cli-listen /run/vpp/cli.sock }
api-trace
{ on }
cpu
{ }
dpdk
{
dev
0000:01:00.1
{ num-rx-queues 1 }
dev
0000:01:00.0
{ num-rx-queues 1 }
}
api-segment
{
gid:
vpp
}
Error log in /var/log/messages after starting VPP
May 16 07:54:06 node6 dbus[6898]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd' May 16 07:54:07 node6 setroubleshoot: SELinux is preventing /usr/bin/vpp from using the chown capability. For complete SELinux messages run: sealert -l 00e1e2d2-4f30-4f38-923c-d34aa07646bf May 16 07:54:07 node6 python: SELinux is preventing /usr/bin/vpp from using the chown capability.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should have the chown capability by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp' --raw | audit2allow -M my-vpp#012# semodule -i my-vpp.pp#012 May 16 07:54:07 node6 setroubleshoot: SELinux is preventing /usr/bin/vpp from setopt access on the netlink_route_socket labeled vpp_t. For complete SELinux messages run: sealert -l feb2ffea-80c5-4804-aeac-c9c42e28448b May 16 07:54:07 node6 python: SELinux is preventing /usr/bin/vpp from setopt access on the netlink_route_socket labeled vpp_t.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed setopt access on netlink_route_socket labeled vpp_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp' --raw | audit2allow -M my-vpp#012# semodule -i my-vpp.pp#012 May 16 07:54:07 node6 setroubleshoot: SELinux is preventing /usr/bin/vpp from create access on the netlink_socket labeled vpp_t. For complete SELinux messages run: sealert -l eee0e1f0-ea9f-4f1d-9ead-9541dccbbd70 May 16 07:54:07 node6 python: SELinux is preventing /usr/bin/vpp from create access on the netlink_socket labeled vpp_t.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed create access on netlink_socket labeled vpp_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp' --raw | audit2allow -M my-vpp#012# semodule -i my-vpp.pp#012 May 16 07:54:07 node6 setroubleshoot: SELinux is preventing /usr/bin/vpp from setopt access on the netlink_socket labeled vpp_t. For complete SELinux messages run: sealert -l d35b5ba0-17be-4852-a27e-2ab50d9ba173 May 16 07:54:07 node6 python: SELinux is preventing /usr/bin/vpp from setopt access on the netlink_socket labeled vpp_t.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed setopt access on netlink_socket labeled vpp_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp' --raw | audit2allow -M my-vpp#012# semodule -i my-vpp.pp#012 May 16 07:54:07 node6 setroubleshoot: SELinux is preventing /usr/bin/vpp from bind access on the netlink_socket labeled vpp_t. For complete SELinux messages run: sealert -l a58fda4d-cda0-4fae-9893-1abecbc51a48 May 16 07:54:07 node6 python: SELinux is preventing /usr/bin/vpp from bind access on the netlink_socket labeled vpp_t.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed bind access on netlink_socket labeled vpp_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp' --raw | audit2allow -M my-vpp#012# semodule -i my-vpp.pp#012 May 16 07:54:10 node6 setroubleshoot: SELinux is preventing /usr/bin/vpp from 'read, write' accesses on the chr_file uverbs0. For complete SELinux messages run: sealert -l b753a050-5190-4ebf-939e-25925c10a912 May 16 07:54:10 node6 python: SELinux is preventing /usr/bin/vpp from 'read, write' accesses on the chr_file uverbs0.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed read write access on the uverbs0 chr_file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp' --raw | audit2allow -M my-vpp#012# semodule -i my-vpp.pp#012 May 16 07:54:10 node6 setroubleshoot: SELinux is preventing /usr/bin/vpp from 'read, write' accesses on the chr_file uverbs0. For complete SELinux messages run: sealert -l b753a050-5190-4ebf-939e-25925c10a912 May 16 07:54:10 node6 python: SELinux is preventing /usr/bin/vpp from 'read, write' accesses on the chr_file uverbs0.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed read write access on the uverbs0 chr_file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp' --raw | audit2allow -M my-vpp#012# semodule -i my-vpp.pp#012 May 16 07:54:19 node6 setroubleshoot: failed to retrieve rpm info for /dev/infiniband/uverbs0 May 16 07:54:19 node6 setroubleshoot: SELinux is preventing vpp from map access on the chr_file /dev/infiniband/uverbs0. For complete SELinux messages run: sealert -l d012cc6d-3a06-4745-90ae-0ff472eba74d May 16 07:54:19 node6 python: SELinux is preventing vpp from map access on the chr_file /dev/infiniband/uverbs0.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed map access on the uverbs0 chr_file by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp' --raw | audit2allow -M my-vpp#012# semodule -i my-vpp.pp#012 May 16 07:54:31 node6 dbus[6898]: [system] Activating service name='org.fedoraproject.Setroubleshootd' (using servicehelper) May 16 07:54:31 node6 dbus[6898]: [system] Successfully activated service 'org.fedoraproject.Setroubleshootd' May 16 07:54:32 node6 setroubleshoot: SELinux is preventing vpp from getattr access on the netlink_route_socket labeled vpp_t. For complete SELinux messages run: sealert -l c2e24f89-1837-47a5-9399-d4c1203416d7 May 16 07:54:32 node6 python: SELinux is preventing vpp from getattr access on the netlink_route_socket labeled vpp_t.#012#012***** Plugin catchall (100. confidence) suggests **************************#012#012If you believe that vpp should be allowed getattr access on netlink_route_socket labeled vpp_t by default.#012Then you should report this as a bug.#012You can generate a local policy module to allow this access.#012Do#012allow this access for now by executing:#012# ausearch -c 'vpp' --raw | audit2allow -M my-vpp#012# semodule -i my-vpp.pp#012
[centos@node6 test]$ sudo ausearch -c 'vpp' --raw | audit2allow -M my-vpp*
To make this policy package active, execute:
semodule -i my-vpp.pp
[centos@node6 test]$ ls
my-vpp.pp my-vpp.te
[centos@node6 test]$ cat my-vpp.te
module my-vpp 1.0;
require
{ type vpp_t; type infiniband_device_t; class capability chown; class netlink_route_socket
{ getattr setopt }
;
class netlink_socket { bind create setopt };
class chr_file { getattr map open read write };
}
#============= vpp_t ==============
allow vpp_t infiniband_device_t:chr_file { getattr map open read write };
allow vpp_t self:capability chown;
allow vpp_t self:netlink_route_socket { getattr setopt };
allow vpp_t self:netlink_socket { bind create setopt };
Assignee
Billy McFall
Reporter
Eyle Brinkhuis
Comments
- ebri (Thu, 23 May 2019 15:53:11 +0000): Billy,
Pardon me for responding this late. It turns out it is my fault, I had forgotten to load ib_uverbs library. Loading this made it work without any issues.
Cheers,
Eyle
- billym (Fri, 17 May 2019 20:45:12 +0000): I got temporary access to a colleagues server with Mellanox ConnectX-4. Built my patch with VPP master (last commit: 0a715cd03fe55719b8bc78cad0cf761df690157a UDP-Local: fix unregistered ports) on CentOS Linux release 7.6.1810, kernel 3.10.0-957.1.3.el7.x86_64, with the additional patch from Benoît Ganne to load RPMs on CentOS: https://gerrit.fd.io/r/c/19561/
With SELinux running (Permissive), I was able to add the interface with no SELinux exceptions and no VPP crash.
$ vppctl create int rdma host-if eno2 name rdma-0 $ vppctl set interface state rdma-0 up $ vppctl set interface ip address rdma-0 1.1.1.2/24 $ vppctl show interface Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count local0 0 down 0/0/0/0 rdma-0 1 up 9000/0/0/0 $ vppctl show pci Address Sock VID:PID Link Speed Driver Product Name Vital Product Data 0000:19:00.0 0 15b3:1015 8.0 GT/s x8 mlx5_core Mellanox ConnectX-4 LX Dual Port PN: 0R887V EC: A04 MN: 1028 SN: CN0R887V7873384MK545 VA: 0x 44 53 56 31 30 32 38 56 ... VB: 0x 46 46 56 31 34 2e 32 31 ... VC: 0x 4e 50 59 32 VD: 0x 50 4d 54 37 38 VE: 0x 4e 4d 56 4d 65 6c 6c 61 ... VF: 0x 44 54 49 4e 49 43 VG: 0x 44 43 4d 31 30 30 31 30 ... VH: 0x 4c 31 44 30 RV: 0x 8c 00 00 0000:19:00.1 0 15b3:1015 8.0 GT/s x8 mlx5_core Mellanox ConnectX-4 LX Dual Port PN: 0R887V EC: A04 MN: 1028 SN: CN0R887V7873384MK545 VA: 0x 44 53 56 31 30 32 38 56 ... VB: 0x 46 46 56 31 34 2e 32 31 ... VC: 0x 4e 50 59 32 VD: 0x 50 4d 54 37 38 VE: 0x 4e 4d 56 4d 65 6c 6c 61 ... VF: 0x 44 54 49 4e 49 43 VG: 0x 44 43 4d 31 30 30 31 30 ... VH: 0x 4c 31 44 30 RV: 0x 8c 00 00
This does not solve the issue. VPP now crashes on creating an RDMA interface, nor are the DPDK interfaces available after installation. The logfile shows:
May 17 14:28:37 node7 vnet[14752]: received signal SIGSEGV, PC 0x7f9883bb9add, faulting address 0x0 May 17 14:28:37 node7 vnet[14752]: #0 0x00007f998d932a18 0x7f998d932a18 May 17 14:28:37 node7 vnet[14752]: #1 0x00007f998d23b5d0 0x7f998d23b5d0 May 17 14:28:37 node7 vnet[14752]: #2 0x00007f9883bb9add 0x7f9883bb9add May 17 14:28:37 node7 vnet[14752]: #3 0x00007f9883ba7a00 rdma_create_if + 0x910 May 17 14:28:37 node7 vnet[14752]: #4 0x00007f9883ba602d 0x7f9883ba602d May 17 14:28:37 node7 vnet[14752]: #5 0x00007f998d8d0076 0x7f998d8d0076 May 17 14:28:37 node7 vnet[14752]: #6 0x00007f998d8d06b7 0x7f998d8d06b7 May 17 14:28:37 node7 vnet[14752]: #7 0x00007f998d8d06b7 0x7f998d8d06b7 May 17 14:28:37 node7 vnet[14752]: #8 0x00007f998d8d0988 vlib_cli_input + 0xa8 May 17 14:28:37 node7 vnet[14752]: #9 0x00007f998d92d0a6 0x7f998d92d0a6 May 17 14:28:37 node7 vnet[14752]: #10 0x00007f998d8e9696 0x7f998d8e9696 May 17 14:28:37 node7 vnet[14752]: #11 0x00007f998cdb5784 0x7f998cdb5784 May 17 14:28:37 node7 systemd: vpp.service: main process exited, code=killed, status=6/ABRT May 17 14:28:37 node7 systemd: Unit vpp.service entered failed state. May 17 14:28:37 node7 systemd: vpp.service failed. May 17 14:28:42 node7 systemd: vpp.service holdoff time over, scheduling restart. May 17 14:28:42 node7 systemd: Stopped Vector Packet Processing Process. May 17 14:28:42 node7 systemd: Starting Vector Packet Processing Process...
Patch marked with -2 until code can be verified.
- billym (Thu, 16 May 2019 16:07:22 +0000): I currently do not have any Mellanox NICs. Can someone test the SELinux Policy changes? I will try to get a patch pushed to gerrit later today, but below is a first pass at what is required in the meantime:
$ git diff extras/selinux/vpp-custom.te diff --git a/extras/selinux/vpp-custom.te b/extras/selinux/vpp-custom.te index 1b6e9c2..7cc2d55 100644 --- a/extras/selinux/vpp-custom.te +++ b/extras/selinux/vpp-custom.te @@ -43,7 +43,7 @@ files_tmp_file(vpp_tmp_t) # vpp local policy #-allow vpp_t self:capability { dac_override ipc_lock setgid sys_rawio net_raw sys_admin net_admin }; # too benevolent +allow vpp_t self:capability { dac_override ipc_lock setgid sys_rawio net_raw sys_admin net_admin chown }; # too benevolent dontaudit vpp_t self:capability2 block_suspend; allow vpp_t self:process { execmem execstack setsched signal }; # too benevolent allow vpp_t self:packet_socket { bind create setopt ioctl map }; @@ -51,7 +51,8 @@ allow vpp_t self:tun_socket { create relabelto relabelfrom }; allow vpp_t self:udp_socket { create ioctl }; allow vpp_t self:unix_dgram_socket { connect create ioctl }; allow vpp_t self:unix_stream_socket { create_stream_socket_perms connectto }; -allow vpp_t self:netlink_route_socket { bind create nlmsg_write read write }; +allow vpp_t self:netlink_route_socket { bind create nlmsg_write read write getattr setopt }; +allow vpp_t self:netlink_socket { bind create setopt };
manage_dirs_pattern(vpp_t, vpp_lib_t, vpp_lib_t) manage_files_pattern(vpp_t, vpp_lib_t, vpp_lib_t) @@ -89,6 +90,7 @@ auth_read_passwd(vpp_t)
corenet_rw_tun_tap_dev(vpp_t)
+dev_rw_infiniband_dev(vpp_t) dev_rw_userio_dev(vpp_t) dev_rw_sysfs(vpp_t) dev_read_cpuid(vpp_t)
Original issue: https://jira.fd.io/browse/VPP-1679
