[opentelemetry-ebpf] Add option to set security context for the kernel collector

[bkimbrough88]

I am trying to use the opentelemetry-ebpf chart to get the kernel collector working with a EKS cluster that has node groups running on Bottlerocket. From what I have read in https://github.com/bottlerocket-os/bottlerocket/issues/3791#issuecomment-1957780069, you must manually set the security context because just setting privileged: true is not respected on Bottlerocket. Would it make sense to allow us to just set the security context ourselves, and just default to privileged: true if it is not specified?

Full log of failing node:

ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
resolving kernel headers...
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
cleaning up stale kprobes...
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
launching kernel collector...
+ exec /srv/kernel-collector --host-distro unknown --kernel-headers-source pre_installed --config-file=/etc/network-explorer/config.yaml --disable-nomad-metadata --warning
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/opt/dynatrace/oneagent-paas/agent/lib64/liboneagentproc.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
could not open bpf map: tgid_info_table, error: Cannot allocate memory
Failed to compile eBPF code for the Linux distro 'unknown' running kernel version 6.1.109.
troubleshoot item bpf_compilation_failed (os=Linux,flavor=unknown,headers_src=pre_installed,kernel=6.1.109): ProbeHandler couldn't load BPFModule: Cannot allocate memory
This usually means that kernel headers weren't installed correctly.
Please reach out to support and include this log in its entirety so we can diagnose and fix
the problem.
In the meantime, please install kernel headers manually on each host before running
the Kernel Collector.
To manually install kernel headers, follow the instructions below:
  - for Debian/Ubuntu based distros, run:
      sudo apt-get install --yes "linux-headers-`uname -r`"
  - for RedHat based distros like CentOS and Amazon Linux, run:
      sudo yum install -y "kernel-devel-`uname -r`"

[crutonjohn]

I'd be happy to get this over the finish line

[diegocejasprieto]

any updates?