tracee icon indicating copy to clipboard operation
tracee copied to clipboard

Tracee-rules error decoding GOB buffer

Open AsafEitani opened this issue 3 years ago • 5 comments

In some cases - which I have successfully reproduced but don;t know the cause yet, tracee-rules gets errors trying to decode tracee-ebpf events encoded in the GOB format.

The following are the errors: 2022/03/29 18:35:59 error while decoding event: gob: duplicate type received 2022/03/29 18:35:59 error while decoding event: gob: unknown type id or corrupted data 2022/03/29 18:35:59 error while decoding event: gob: unknown type id or corrupted data 2022/03/29 18:35:59 error while decoding event: gob: encoded unsigned integer out of range 2022/03/29 18:35:59 error while decoding event: unexpected EOF 2022/03/29 18:35:59 error while decoding event: gob: encoded unsigned integer out of range

I presume it has something to do with the recent change of the Event struct (addition of the protocol and source of the event) since the bug wasn't present in the 0.6.5 version, and was introduced somewhere in between v0.6.5 to v0.7.0-rc-1.

AsafEitani avatar Mar 29 '22 17:03 AsafEitani

Are you using the --debug flag by any chance? We've seen similar error when tracee-ebpf was run with --metrics or --debug flag that clutter the standard output. See for example #1581

danielpacak avatar Mar 29 '22 20:03 danielpacak

Are you using the --debug flag by any chance? We've seen similar error when tracee-ebpf was run with --metrics or --debug flag that clutter the standard output. See for example #1581

Nope, running the following command:

tracee-ebpf -b 1024 --blob-perf-buffer-size 1024 -t event=init_namespaces,switch_task_ns,commit_creds,execve,init_module,magic_write,mem_prot_alert,process_vm_writev,ptrace,sched_process_exec,sched_process_exit,sched_process_fork,security_bpf,security_bprm_check,security_file_open,security_inode_unlink,security_kernel_read_file,security_sb_mount,security_socket_accept,security_socket_bind,security_socket_connect,socket_dup,clock_settime,container_create,container_remove,existing_container -o option:detect-syscall -o option:exec-env -o option:exec-hash -o option:parse-arguments -o option:sort-events -o format:gob -o out-file:ebpf_events.gob -o err-file:ebpf_error.log

^C

tracee-rules --input-tracee=file:ebpf_events.gob --input-tracee=format:gob --output-template=../cmd/tracee-rules/templates/rawjson.tmpl 2> rules_error.log 

AsafEitani avatar Mar 30 '22 07:03 AsafEitani

Couldn't it be because of errors being printed to stdout ? One example is:

$ sudo ./dist/tracee-ebpf --output none --trace comm=ping --capture net=lo
libbpf: Kernel error message: Exclusivity flag on, cannot modify

Already flagged as an issue (has to be filtered in libbpf print out function).

rafaeldtinoco avatar Mar 31 '22 12:03 rafaeldtinoco

@AsafEitani do you have a reproducer here ? How are running tracee to get this error ? (Cmdline, so I can try it if you don't have a reproducer).

rafaeldtinoco avatar Apr 01 '22 11:04 rafaeldtinoco

@rafaeldtinoco see this comment:

Are you using the --debug flag by any chance? We've seen similar error when tracee-ebpf was run with --metrics or --debug flag that clutter the standard output. See for example #1581

Nope, running the following command:

tracee-ebpf -b 1024 --blob-perf-buffer-size 1024 -t event=init_namespaces,switch_task_ns,commit_creds,execve,init_module,magic_write,mem_prot_alert,process_vm_writev,ptrace,sched_process_exec,sched_process_exit,sched_process_fork,security_bpf,security_bprm_check,security_file_open,security_inode_unlink,security_kernel_read_file,security_sb_mount,security_socket_accept,security_socket_bind,security_socket_connect,socket_dup,clock_settime,container_create,container_remove,existing_container -o option:detect-syscall -o option:exec-env -o option:exec-hash -o option:parse-arguments -o option:sort-events -o format:gob -o out-file:ebpf_events.gob -o err-file:ebpf_error.log

^C

tracee-rules --input-tracee=file:ebpf_events.gob --input-tracee=format:gob --output-template=../cmd/tracee-rules/templates/rawjson.tmpl 2> rules_error.log 

It has the command line used. It seems that this error occurs in almost every event - even if run with a specific filter for that event.

AsafEitani avatar Apr 03 '22 13:04 AsafEitani

Solved by #2160

AsafEitani avatar Sep 19 '22 09:09 AsafEitani