IPIPDirect-TC icon indicating copy to clipboard operation
IPIPDirect-TC copied to clipboard

update

Open dedimark opened this issue 1 year ago • 7 comments
trafficstars

uname -a 6.1.0-20-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.85-1 (2024-04-11) x86_64 GNU/Linux

lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 12 (bookworm) Release: 12 Codename: bookworm

/usr/bin/IPIPDirect_loader eth0 libbpf: elf: legacy map definitions in 'maps' section are not supported by libbpf v1.0+ ERROR: opening BPF object file failed Unable to load program Error attaching TC egress filter. TC cannot attach to filter. Command => tc filter add dev eth0 egress prio 1 handle 1 bpf da obj /etc/IPIPDirect/IPIPDirect_filter.o sec egress and Return Error Number => 1.

https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#drop-support-for-legacy-bpf-map-declaration-syntax

can you help recompile on newest OS?

dedimark avatar Apr 26 '24 08:04 dedimark

Hey, are you using the correct version of LibBPF that I used with this project? You should have the following commit checked out under src/include/libbpf.

https://github.com/libbpf/libbpf/tree/97ada10bd834950b1856c7f9efacc9ee83f824cf

In regards to porting this project to the latest version of LibBPF, I might do that in the future, but I'm not sure when I'll have the time.

gamemann avatar Apr 27 '24 17:04 gamemann

yes, libbpf correct version.

git clone --recursive https://github.com/gamemann/IPIPDirect-TC.git Cloning into 'IPIPDirect-TC'... remote: Enumerating objects: 85, done. remote: Counting objects: 100% (85/85), done. remote: Compressing objects: 100% (59/59), done. remote: Total 85 (delta 39), reused 70 (delta 24), pack-reused 0 Receiving objects: 100% (85/85), 271.87 KiB | 3.02 MiB/s, done. Resolving deltas: 100% (39/39), done. Submodule 'src/include/libbpf' (https://github.com/libbpf/libbpf.git) registered for path 'src/include/libbpf' Cloning into '/root/IPIPDirect-TC/src/include/libbpf'... remote: Enumerating objects: 12184, done. remote: Counting objects: 100% (3393/3393), done. remote: Compressing objects: 100% (858/858), done. remote: Total 12184 (delta 2539), reused 2622 (delta 2450), pack-reused 8791 Receiving objects: 100% (12184/12184), 9.21 MiB | 18.67 MiB/s, done. Resolving deltas: 100% (8197/8197), done. Submodule path 'src/include/libbpf': checked out '97ada10bd834950b1856c7f9efacc9ee83f824cf'

dedimark avatar Apr 29 '24 09:04 dedimark

Hey,any news?

dedimark avatar Jun 30 '24 04:06 dedimark

Hi, I've pushed an update to the project that should work with the latest version of LibBPF.

I wasn't able to test the TC program's functionality itself since I no longer run this in production, but confirmed attaching/detaching from the TC egress filter works.

# TC filter egress programs loaded.
christian@dev-debian12:~$ sudo tc filter show dev enp1s0 egress
filter protocol all pref 49152 bpf chain 0 
filter protocol all pref 49152 bpf chain 0 handle 0x1 tc_egress:[399] direct-action not_in_hw id 399 name tc_egress tag 245416d498d264d0 jited 

# Ensuring mac_map is populated meaning new map pinning works.
christian@dev-debian12:~$ sudo bpftool map dump name mac_map
[{
        "key": 0,
        "value": 156300082273574
    }
]

Please let me know how it goes and ensure you update LibBPF to the latest version when pulling the new changes.

gamemann avatar Jun 30 '24 21:06 gamemann

git clone --recursive https://github.com/gamemann/IPIPDirect-TC.git

Cloning into 'IPIPDirect-TC'... remote: Enumerating objects: 119, done. remote: Counting objects: 100% (119/119), done. remote: Compressing objects: 100% (77/77), done. remote: Total 119 (delta 57), reused 100 (delta 38), pack-reused 0 Receiving objects: 100% (119/119), 277.08 KiB | 3.96 MiB/s, done. Resolving deltas: 100% (57/57), done. Submodule 'src/include/libbpf' (https://github.com/libbpf/libbpf.git) registered for path 'src/include/libbpf' Cloning into '/root/IPIPDirect-TC/src/include/libbpf'... remote: Enumerating objects: 12602, done. remote: Counting objects: 100% (3811/3811), done. remote: Compressing objects: 100% (985/985), done. remote: Total 12602 (delta 2838), reused 2949 (delta 2720), pack-reused 8791 Receiving objects: 100% (12602/12602), 9.60 MiB | 16.75 MiB/s, done. Resolving deltas: 100% (8496/8496), done. Submodule path 'src/include/libbpf': checked out 'c1a6c770c46c6e78ad6755bf596c23a4e6f6b216'

root@vultr:~/IPIPDirect-TC# make

make[1]: Leaving directory '/root/IPIPDirect-TC/src/include/libbpf/src' clang -lelf -lz -o build/IPIPDirect_loader src/include/libbpf/src/staticobjs/bpf_prog_linfo.o src/include/libbpf/src/staticobjs/bpf.o src/include/libbpf/src/staticobjs/btf_dump.o src/include/libbpf/src/staticobjs/btf_iter.o src/include/libbpf/src/staticobjs/btf_relocate.o src/include/libbpf/src/staticobjs/btf.o src/include/libbpf/src/staticobjs/elf.o src/include/libbpf/src/staticobjs/features.o src/include/libbpf/src/staticobjs/gen_loader.o src/include/libbpf/src/staticobjs/hashmap.o src/include/libbpf/src/staticobjs/libbpf_errno.o src/include/libbpf/src/staticobjs/libbpf_probes.o src/include/libbpf/src/staticobjs/libbpf.o src/include/libbpf/src/staticobjs/linker.o src/include/libbpf/src/staticobjs/netlink.o src/include/libbpf/src/staticobjs/nlattr.o src/include/libbpf/src/staticobjs/relo_core.o src/include/libbpf/src/staticobjs/ringbuf.o src/include/libbpf/src/staticobjs/str_error.o src/include/libbpf/src/staticobjs/strset.o src/include/libbpf/src/staticobjs/usdt.o src/include/libbpf/src/staticobjs/zip.o src/IPIPDirect_loader.c clang -O2 --target=bpf -g -c src/IPIPDirect_kern.c -o build/IPIPDirect_filter.o In file included from src/IPIPDirect_kern.c:14: src/include/bpf_helpers.h:4:10: fatal error: 'bpf/bpf_helpers.h' file not found #include <bpf/bpf_helpers.h> ^~~~~~~~~~~~~~~~~~~ 1 error generated. make: *** [Makefile:22: kern] Error 1

I can't compile

src/include/bpf_helpers.h:4:10: fatal error: 'bpf/bpf_helpers.h' file not found #include <bpf/bpf_helpers.h>

dedimark avatar Jul 17 '24 11:07 dedimark

I'll look into this later. It likely works on my VM because I've installed xdp-tools on the system. You could try installing that along with LibBPF to see if it corrects the error you're getting in the meantime.

gamemann avatar Jul 21 '24 23:07 gamemann

uname -a Linux vultr 6.1.0-22-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.94-1 (2024-06-21) x86_64 GNU/Linux

apt-get install xdp-tools Reading package lists... Done Building dependency tree... Done Reading state information... Done xdp-tools is already the newest version (1.3.1-1).

--

make make -C src/include/libbpf/src make[1]: Entering directory '/root/IPIPDirect-TC/src/include/libbpf/src' make[1]: Leaving directory '/root/IPIPDirect-TC/src/include/libbpf/src' clang -lelf -lz -o build/IPIPDirect_loader src/include/libbpf/src/staticobjs/bpf_prog_linfo.o src/include/libbpf/src/staticobjs/bpf.o src/include/libbpf/src/staticobjs/btf_dump.o src/include/libbpf/src/staticobjs/btf_iter.o src/include/libbpf/src/staticobjs/btf_relocate.o src/include/libbpf/src/staticobjs/btf.o src/include/libbpf/src/staticobjs/elf.o src/include/libbpf/src/staticobjs/features.o src/include/libbpf/src/staticobjs/gen_loader.o src/include/libbpf/src/staticobjs/hashmap.o src/include/libbpf/src/staticobjs/libbpf_errno.o src/include/libbpf/src/staticobjs/libbpf_probes.o src/include/libbpf/src/staticobjs/libbpf.o src/include/libbpf/src/staticobjs/linker.o src/include/libbpf/src/staticobjs/netlink.o src/include/libbpf/src/staticobjs/nlattr.o src/include/libbpf/src/staticobjs/relo_core.o src/include/libbpf/src/staticobjs/ringbuf.o src/include/libbpf/src/staticobjs/str_error.o src/include/libbpf/src/staticobjs/strset.o src/include/libbpf/src/staticobjs/usdt.o src/include/libbpf/src/staticobjs/zip.o src/IPIPDirect_loader.c clang -O2 --target=bpf -g -c src/IPIPDirect_kern.c -o build/IPIPDirect_filter.o In file included from src/IPIPDirect_kern.c:14: src/include/bpf_helpers.h:4:10: fatal error: 'bpf/bpf_helpers.h' file not found #include <bpf/bpf_helpers.h> ^~~~~~~~~~~~~~~~~~~ 1 error generated. make: *** [Makefile:22: kern] Error 1

same.

dedimark avatar Jul 22 '24 21:07 dedimark

Hey,still need help.

dedimark avatar Sep 11 '24 15:09 dedimark

Hey, please pull the latest changes and try again. The program builds successfully on a new stock Debian 12 VM I created after installing the following packages via apt.

sudo apt install -y build-essential make libelf-dev clang gcc-multilib

gamemann avatar Sep 13 '24 11:09 gamemann

compile successfully,but

root@vultr:~/IPIPDirect-TC# /usr/bin/IPIPDirect_loader enp1s0 libbpf: failed to statfs /sys/fs/bpf/tc/globals: No such file or directory libbpf: failed to mkdir /sys/fs/bpf/tc/globals/mac_map: No such file or directory libbpf: Kernel error message: Exclusivity flag on, cannot modify Error getting map. Map name => /sys/fs/bpf/tc/globals/mac_map

git clone --recursive https://github.com/gamemann/IPIPDirect-TC.git
Cloning into 'IPIPDirect-TC'...
remote: Enumerating objects: 127, done.
remote: Counting objects: 100% (127/127), done.
remote: Compressing objects: 100% (82/82), done.
remote: Total 127 (delta 62), reused 106 (delta 41), pack-reused 0 (from 0)
Receiving objects: 100% (127/127), 277.74 KiB | 2.92 MiB/s, done.
Resolving deltas: 100% (62/62), done.
Submodule 'src/include/libbpf' (https://github.com/libbpf/libbpf.git) registered for path 'src/include/libbpf'
Cloning into '/root/IPIPDirect-TC/src/include/libbpf'...
remote: Enumerating objects: 12765, done.
remote: Counting objects: 100% (3975/3975), done.
remote: Compressing objects: 100% (1008/1008), done.
remote: Total 12765 (delta 2957), reused 3093 (delta 2855), pack-reused 8790 (from 1)
Receiving objects: 100% (12765/12765), 9.68 MiB | 10.63 MiB/s, done.
Resolving deltas: 100% (8615/8615), done.
root@vultr:~/IPIPDirect-TC# apt-get install xdp-tools
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
xdp-tools is already the newest version (1.3.1-1).
root@vultr:~/IPIPDirect-TC# sudo apt install -y build-essential make libelf-dev clang gcc-multilib git
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
build-essential is already the newest version (12.9).
make is already the newest version (4.3-4.1).
libelf-dev is already the newest version (0.188-2.1).
clang is already the newest version (1:14.0-55.7~deb12u1).
gcc-multilib is already the newest version (4:12.2.0-3).
git is already the newest version (1:2.39.2-1.1).
uname -a
Linux vultr 6.1.0-25-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.106-3 (2024-08-26) x86_64 GNU/Linux
lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)

dedimark avatar Sep 16 '24 14:09 dedimark

Ah, I had to mount BPF and create the tc/globals directory for the program to load properly.

sudo mount -t bpf bpf /sys/fs/bpf
sudo mkdir -p /sys/fs/bpf/tc/globals

Outputs showing it works as expected.

christian@dev-debian12:~/IPIPDirect-TC$ sudo IPIPDirect_loader enp1s0
libbpf: Kernel error message: Exclusivity flag on, cannot modify
Starting IPIP Direct TC egress program.
christian@dev-debian12:~$ sudo tc filter show dev enp1s0 egress
filter protocol all pref 49152 bpf chain 0 
filter protocol all pref 49152 bpf chain 0 handle 0x1 tc_egress:[197] direct-action not_in_hw id 197 name tc_egress tag 245416d498d264d0 jited 

The functionality should work, but I haven't tested the program after my recent commits. As long as the map pinning works as expected, there shouldn't be any issues.

Also, you don't need xdp-tools for this. I thought that would have corrected your previous issue, but since it didn't, you do not need it.

gamemann avatar Sep 16 '24 22:09 gamemann

Thank you,

I'll reinstall OS on game servers to Debian 12 and let u know.

dedimark avatar Sep 18 '24 23:09 dedimark