bpftime icon indicating copy to clipboard operation
bpftime copied to clipboard

[FEATURE] Add trace_pipe support for bpf_printk

Open Officeyutong opened this issue 1 year ago • 9 comments

The bpf_prink helper currently output the results to the target program. This is different from the kernel eBPF behavior and may break the target program.

We may need a feature that allow agent to send the output of bpf_printk to shm and buffer them, so that we could see logs using bpftime cli, or display them when using remote attach.

Officeyutong avatar Jan 27 '24 06:01 Officeyutong

I think the bpf_printk output should not be sent to the shm. We can use a name pipe.

It can be similar like kernel /sys/kernel/debug/tracing/trace_pipe

The cli can read the data from the pipe, and the pipe can also be shared between processes.

yunwei37 avatar Jan 27 '24 14:01 yunwei37

And maybe all the log from bpftime runtime should also be sent to the name pipe for debug.

yunwei37 avatar Jan 27 '24 15:01 yunwei37

+1. I hope the bpf_printk's output goes to the stderr stream of the bpftime load program so that it's easier to separate the debugging output from the normal output data currently going to stdout.

agentzh avatar Jan 27 '24 20:01 agentzh

Hello, I am new to the code, and had a doubt after reading the issue comments and the code - Will the cli have a new option for reading the data from the pipe?

aneeshdamle11 avatar Mar 02 '24 12:03 aneeshdamle11

Hello, I am new to the code, and had a doubt after reading the issue comments and the code - Will the cli have a new option for reading the data from the pipe?

Yes

Officeyutong avatar Mar 02 '24 12:03 Officeyutong

I think it is an interesting issue, and would love to explore more. Also, would the implementation of bpf_printk change, or will there be a new function to write in the tracepipe?

aneeshdamle11 avatar Mar 02 '24 18:03 aneeshdamle11

I think it is an interesting issue, and would love to explore more. Also, would the implementation of bpf_printk change, or will there be a new function to write in the tracepipe?

bpf_printk should keep its current function, but we could use attach config (such as an option when invoking bpftime attach) to make bpf_printk output things to the pipe

Officeyutong avatar Mar 03 '24 04:03 Officeyutong

I read more on the issue and referring to: https://eunomia.dev/bpftime/documents/usage/#uprobe-and-uretprobe, bpf_printk's result is shown in the output of the target (victim) program a) when it is called with the start option, or b) when the attach option works on the PID. Hence, start and attach are responsible for dealing with bpf_printk's results. I read some corresponding code and tried to identify the areas where tracelog is being read. Could you please help with any areas in the code I could concentrate on further?

aneeshdamle11 avatar Mar 11 '24 18:03 aneeshdamle11

I read more on the issue and referring to: https://eunomia.dev/bpftime/documents/usage/#uprobe-and-uretprobe, bpf_printk's result is shown in the output of the target (victim) program a) when it is called with the start option, or b) when the attach option works on the PID. Hence, start and attach are responsible for dealing with bpf_printk's results. I read some corresponding code and tried to identify the areas where tracelog is being read. Could you please help with any areas in the code I could concentrate on further?

bpf_trace_printk (which is the function that bpf_printk internally calls), is a bpf helper implemented at https://github.com/eunomia-bpf/bpftime/blob/103903b139a9c77a7edb14866fd8a7c763385d4d/runtime/src/bpf_helper.cpp#L43, in the bpftime agent runtime.

In both bpftime start and bpftime attach, the agent was injected to the process that we want to monitor. So in the current implementation, call to bpf_trace_printk will result a message printed to stdout, at the process that was injected with agent

Officeyutong avatar Mar 12 '24 11:03 Officeyutong