perf: add support to collect hardware counters

[marceloamaral]

Describe the bug Currently, as far as I understood, the library only support software PMUs, as here

Expected behavior Create a perfEvent with to read hardware counter,:

config := unix. PERF_COUNT_HW_CACHE_MISSES
attr := unix.PerfEventAttr{
		Type:        unix.ERF_TYPE_HARDWARE,
		Config:     config,
	}

[lmb]

Others want to use the perf reader for more advanced use case as well. Please take a look at https://github.com/cilium/ebpf/pull/818#issuecomment-1316890797 for what I think would need to happen.

[florianl]

Just my personal point of view and also with https://github.com/cilium/ebpf/pull/818#issuecomment-1316890797 in mind: perf and ebpf are two very different subsystems of the Linux kernel. Both together enable very interesting use cases. But on the same time both subsystem are complex and implementing their userspace component is not trivial. What comes next after implementing and maintaining PERF_TYPE_SOFTWARE and PERF_TYPE_HARDWARE within the scope of cilium/ebpf? PERF_TYPE_HARDWARE on its own can become very complex when providing a proper implementation and taking the vendor specific parts into account.

Looking at acln0/perf, hodgesds/perf-utils and other Go packages that implement the userspace part of the perf subsystem, I think there are alternatives. And once one can get a file descriptor for a perf event, it is a matter of SYS_IOCTL to attach an eBPF program to it.

Overall I do think there is a need for examples how cilium/ebpf can be used along with such packages. Requests for examples for XDP comes into my mind as well. But as mentioned in another place, I also do see cilium/ebpf/examples more of a place to showcase the functionality of cilium/ebpf ifself. Maybe opening the wiki of cilium/ebpf for such examples with third party packages would be an option?

[lmb]

To clarify, I wouldn't want to support these distinct sample types directly, that is still up to the user. However it seems that people want to reuse the reader implementation from this library instead of the dedicated perf packages. Maybe @olsajiri or @marceloamaral can explain why the other package's readers aren't suitable?

[marceloamaral]

We can't create custom metrics with perf, ebpf is more flexible. So perf is only useful if someone wants to use the predefined metrics, but there are other cases where people want to create their own metrics. cilium/ebpf could be a generic library that other projects could use.

[lmb]

Seems to me like at least go-perf supports reading raw events from an event: https://pkg.go.dev/github.com/elastic/go-perf#RawRecord What's missing to use this from ebpf?

[olsajiri]

so for tetragon the ebpf/cilium is already used to read the perf ring buffer for bpf-ouput events and we want to enable other types of events on the same ring buffer.. having some general perf reader would be great, so we don't need to import another package and have separate ring buffers

[lmb]

So basically it's not clear how to integrate go-perf with a PerfEventArray? What about the following:

1. Create perf events with your custom config, one per CPU. This could include non-BPF sample types. (I think this is what #818 is about)
2. Do PerfEventArray.Put(cpu, event)
3. Read from the events

Step two is the only really BPF specific thing I can see to be honest. Maybe it's enough if we make the logic for this from NewReaderWithOptions available somehow?

having some general perf reader would be great, so we don't need to import another package

Wouldn't you use the same package for step 1 and 3?

... and have separate ring buffers

Seems like with the scheme above that could be avoided?

[olsajiri]

1. Create perf events with your custom config, one per CPU. This could include non-BPF sample types. (I think this is what [ebpf: Allow to pass custom attr to perf ring reader #818](https://github.com/cilium/ebpf/pull/818) is about)

2. Do PerfEventArray.Put(cpu, event)

3. Read from the events

that seems much better than other factorings I've tried so far ;-)

Step two is the only really BPF specific thing I can see to be honest. Maybe it's enough if we make the logic for this from NewReaderWithOptions available somehow?

how about option that says the PerfEventArray is already loaded so the reader won't create new perf events and update the map with them

having some general perf reader would be great, so we don't need to import another package

Wouldn't you use the same package for step 1 and 3?

... and have separate ring buffers

Seems like with the scheme above that could be avoided?

I'd like to use just ebpf/cilium because that's what we use now, scheme above seems good for what we need

thanks

[lmb]

Not going to do this in c/ebpf. The right place is a separate perf package which we could depend on.