SysmonForLinux icon indicating copy to clipboard operation
SysmonForLinux copied to clipboard

sysmon.service won't start because of libbpf issue

Open luffynextgen opened this issue 3 years ago • 10 comments

OS and Sysmon version Info

CentOS 8.2.2004 Kernel Version 4.18.0-193.19.1.el8_2.x86_64 sysmonforlinux-1.0.2-1.x86_64 packages-microsoft-prod-1.0.1.noarch

Error printed

When using sysmon -accepteula -i

  • Job for sysmon.service failed because the control process exited with error code
  • Details in service status:
sysmon_service_details

The issue seems to come from sysinternalsEPBF. The package was downloaded from the yum repository.

Note

sysmonforlinux works well on CentOS 8.1.1911 (Core) with kernel 4.18.0-147.5.1.el8_1.x86_64 EDIT: I tried to restart the sysmon service on the server it was working, and now it won't restart:

   1. first the error I had was about  libsysinternalsEBPF.so that was not found, I had to recompile it to solve this issue
   2. Once the library was recompiled and found, now sysmon can't find sysinternalsEBPFrawSock.o program
   sysinternalsEBPFrawSock.o program is well located in  /opt/sysinternalsEBPF/ but sysmon can't find it:
   
   Apr 12 16:18:32 server01 sysmon[222692]: Using EBPF object: .//sysmonEBPFkern4.17-5.1.o
   Apr 12 16:18:34 server01 sysmon[222692]: ERROR: failed to locate program: /opt/sysinternalsEBPF/sysinternalsEBPFrawSock.o 'Invalid argument'
   Apr 12 16:18:34 server01 sysmon[222692]: ERROR: failed to enable raw socket capture
   Apr 12 16:18:34 server01 sysmon[222692]: Telemetry failed to start: Raw socket program could not be attached

luffynextgen avatar Apr 12 '22 08:04 luffynextgen

Almost samething with Debian OS - sysmon service doesnt start. My specification:

OS: Debian 10.12.0 Kernel: Linux 4.19.0-20-amd64 sysmonforlinux-1.0.2

sysmon service status error:

sysmon_err

sysmon accepteula:

sysmon_err1

anforcer avatar Apr 15 '22 04:04 anforcer

On the CentOS server 8.1.1911 (Core) with kernel 4.18.0-147.5.1.el8_1.x86_64, I update sysmon to version 1.0.2-1 This updates give the same math issue as the one on the CentOS 8.2.2004 Server or by @anforcer with the Debian server. image

Kind regards,

luffynextgen avatar Apr 26 '22 08:04 luffynextgen

FWIW I'm having same experiments with other OS - Fedora 34 worked no issues for me but RHEL8.4 and then an upgrade to RHEL8.5 encountered same problems (I tried an install of both sysmon and sysinternalsebpf 1.0.0-1 - different error but won't install) This is from the RHEL8.5 installing latest sysmon.

image

BDH-Granicus avatar Apr 28 '22 17:04 BDH-Granicus

Hi @kesheldr, Sorry for the direct tagging... Can you provide some hints for solving this issue? Thanks in advance!

raomin avatar May 02 '22 09:05 raomin

It looks like the verifier is having problems verifying the eBPF program. We'll add this to our backlog to look into.

MarioHewardt avatar May 13 '22 15:05 MarioHewardt

Thanks @MarioHewardt

BDH-Granicus avatar May 14 '22 17:05 BDH-Granicus

Thanks for your support @MarioHewardt. Any plan on a release date for this?

raomin avatar Jun 01 '22 09:06 raomin

Sorry for the delay. I'm working on some high priority release related work items at the moment. Once done, investigating these RHEL/CentOS issues will be up next.

MarioHewardt avatar Jun 01 '22 16:06 MarioHewardt

Hello @MarioHewardt, this is still preventing us from deploying Sysmon on our linux servers. Would you have an idea on when we can expect a solution? Thanks!

raomin avatar Jul 01 '22 07:07 raomin

I'm still working on some other work items. I will update everyone as soon as I know more. Thanks for the patience.

MarioHewardt avatar Jul 01 '22 23:07 MarioHewardt

Closing as I've pushed a fix that should resolve the issue. If you encounter it again, please reopen. You may have to run getOffsets (https://github.com/Sysinternals/SysinternalsEBPF/tree/main/getOffsets) to get this to work on CentOS8.

Please note that you will have to build Sysmon until we get new packages out.

MarioHewardt avatar Jan 20 '23 01:01 MarioHewardt