Running./tracee has a huge impact on the performance of the system call 'write'
Description
Run the following commands, respectively, and monitor the performance impact of system calls 1、./tracee 2、./tracee --signatures-dir (Specify an empty folder here) 3、./tracee-ebpf The time taken for system calls when tracee is not running is used as a benchmark,The impact of the above three scenarios on system calls (‘write’)is 1、21.93 --This means that after running tracee, the latency of the write system call is 21.93 times higher than without running tracee 2、1.86 3、1.86
Output of tracee version:
0.17.1
Output of uname -a:
4.18.0-372.26.1
Additional details
When running./tracee, whether or not default signatures are loaded has such a big impact on the system call ‘’write‘. Why?
This is indeed a very high latency, and I want to first understand how to reproduce these results. If I have to guess, I would say the reason for this latency is one of the signature events that depends on vfs_write under the hood. For example, the dropped_executable signature depends on the magic_write event which is generated by the bpf program attached to vfs_write. This program extracts the full file path on every call, which can be expensive.
Can you please share with us how you measure the latency for the write syscall? Which tool do you use to measure? What is the load you run that generates the syscalls? What is the file path being written (especially interesting if the path has many components)? Are the writes being performed in some loop? Is it disk io or memory io? Any other details you can share about the environment?
@AlonZivony were you able to look into this?
@AlonZivony were you able to look into this?
No, I am focusing on the process tree currently. I volunteered to fix it, but don't have the tools to check performance. As I said when I took the task, I need someone to help me with this task.
This has been resolved in #3899