deepflow icon indicating copy to clipboard operation
deepflow copied to clipboard

Published 20 hours ago • verified publisher icon deepflowio

[BUG] agent coredump

Open zhiqunlin opened this issue 5 months ago • 1 comments

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

DeepFlow Component

Agent

What you expected to happen

agent cordump

[Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/libthread_db.so.1".

warning: Cannot parse .gnu_debugdata section; LZMA support was disabled at compile time --Type <RET> for more, q to quit, c to continue without paging-- Core was generated by `deepflow-agent'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x000055be236180c9 in hashbrown::map::HashMap<K,V,S,A>::insert () [Current thread is 1 (Thread 0x7f41d25fe700 (LWP 23582))] (gdb) bt full #0 0x000055be236180c9 in hashbrown::map::HashMap<K,V,S,A>::insert () No symbol table info available. #1 0x000055be23943181 in deepflow_agent::policy::labeler::Labeler::update_interface_table () No symbol table info available. #2 0x000055be2394642c in <deepflow_agent::policy::policy::PolicySetter as deepflow_agent::common::FlowAclListener>::flow_acl_change () No symbol table info available. #3 0x000055be2394b8bd in deepflow_agent::rpc::synchronizer::Synchronizer::on_response () No symbol table info available. #4 0x000055be239c4302 in deepflow_agent::rpc::synchronizer::Synchronizer::run_triggered_session::{{closure}} () No symbol table info available. #5 0x000055be239b11af in tokio::runtime::task::core::Core<T,S>::poll () No symbol table info available. #6 0x000055be2372b34c in tokio::runtime::task::harness::Harness<T,S>::poll () No symbol table info available. #7 0x000055be24585081 in tokio::runtime::scheduler::multi_thread::worker::Context::run_task () No symbol table info available. #8 0x000055be2458466f in tokio::runtime::scheduler::multi_thread::worker::Context::run () --Type <RET> for more, q to quit, c to continue without paging-- No symbol table info available. #9 0x000055be2457e879 in tokio::runtime::context::set_scheduler () No symbol table info available. #10 0x000055be245879e4 in tokio::runtime::context::runtime::enter_runtime () No symbol table info available. #11 0x000055be24583fbb in tokio::runtime::scheduler::multi_thread::worker::run () No symbol table info available. #12 0x000055be24581b6a in tokio::runtime::task::core::Core<T,S>::poll () No symbol table info available. #13 0x000055be2456b807 in tokio::runtime::task::harness::Harness<T,S>::poll () No symbol table info available. #14 0x000055be24579fc6 in std::sys_common::backtrace::__rust_begin_short_backtrace () No symbol table info available. #15 0x000055be2457df5d in core::ops::function::FnOnce::call_once{{vtable-shim}} () No symbol table info available. #16 0x000055be246278c5 in std::sys::pal::unix::thread::Thread::new::thread_start () No symbol table info available. #17 0x00007f41ffffcf57 in start_thread (arg=) at pthread_create.c:477 --Type <RET> for more, q to quit, c to continue without paging-- ret = pd = unwind_buf = {cancel_jmp_buf = {{jmp_buf = {139920679102208, 5770730337691132541, 139920702104894, 139920702104895, 139920679098240, 139920679102208, -5866391421952040323, -5866431828195515779}, mask_was_saved = 0}}, priv = {pad = { 0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = 0 #18 0x00007f41ff9f62af in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 No locals. (gdb)

How to reproduce

server端的agent-group-config.yaml配置如下

DeepFlow version

都是6.6.3

eaf64818a286 registry.cn-hongkong.aliyuncs.com/deepflow-ce/deepflow-server:v6.6.3 "/bin/sh -c /bin/dee…" 2 weeks ago Up 5 days 0.0.0.0:20416->20416/tcp, :::20416->20416/tcp, 0.0.0.0:20419->20419/tcp, :::20419->20419/tcp, 0.0.0.0:30033->20033/tcp, :::30033->20033/tcp, 0.0.0.0:30035->20035/tcp, :::30035->20035/tcp, 0.0.0.0:30417->20417/tcp, :::30417->20417/tcp deepflow-server

/data # deepflow-agent -v 11010-540e0d98c6d752605ddb6304138948541a66b01e Name: deepflow-agent community edition Branch: v6.6.3 CommitId: 540e0d98c6d752605ddb6304138948541a66b01e RevCount: 11010 Compiler: rustc 1.77.1 (7cf61ebde 2024-03-27) CompileTime: 2024-08-16 02:07:51

DeepFlow agent list

root@zenosic:/home/deepflow/server/test/deepflow-docker-compose# deepflow-ctl agent list 2024/09/04 09:22:42 open /run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory ID NAME TYPE CTRL_IP CTRL_MAC STATE GROUP EXCEPTIONS REVISION UPGRADE_REVISION

10 (none)-W4 CHOST_VM 10.110.75.212 fa:16:3e:5c:c0:c6 NORMAL legacy-host

Kubernetes CNI

vm虚拟机

Operation-System/Kernel version

agent虚拟机：NOS系统。 /data # awk -F '=' '/PRETTY_NAME/ { print $2 }' /etc/os-release "Buildroot 2017.02.5" /data # uname -r 5.4.241+

server：虚拟机 root@zenosic:/home/deepflow/server/test/deepflow-docker-compose# awk -F '=' '/PRETTY_NAME/ { print $2 }' /etc/os-release "Ubuntu 20.04 LTS" root@zenosic:/home/deepflow/server/test/deepflow-docker-compose# uname -r 5.4.0-153-generic

Anything else

deepflow-ctl agent-group-config update -f agent-group-config.yaml aagent运行的log如下、 10.110.75.212_2024-09-04_17_14_21.log server端的agent-group-config.yaml配置如下：

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

zhiqunlin avatar Sep 04 '24 09:09 zhiqunlin