Bug in latest master

[micron10]

Hi please check this report

[2023-09-30 16:46:53,336][ERROR][ dns_client.c:2635] recv failed, server 8.8.4.4:853, Bad address [2023-09-30 16:46:53,528][ERROR][ dns_cache.c:87 ] BUG: dns_cache is invalid. [2023-09-30 16:46:53,528][FATAL][ smartdns.c:616 ] process exit with signal 6, code = -6, errno = 0, pid = 1138503, self = 1138503, pc = 0x115f47, addr = 0x115f47, build(Sep 27 2023 19:21:19 x86_64) [2023-09-30 16:46:53,528][FATAL][ util.c:1443] Stack: [2023-09-30 16:46:53,528][FATAL][ util.c:1455] #01: 0x42ab9d () from /usr/sbin/smartdns+0x2ab9d [2023-09-30 16:46:53,528][FATAL][ util.c:1455] #02: 0x40637d () from /usr/sbin/smartdns+0x637d [2023-09-30 16:46:53,528][FATAL][ util.c:1455] #03: 0x1491cc546d30 () from /lib64/libc.so.6+0x40d30 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #04: 0x1491cc59ac0b pthread_kill() from /lib64/libc.so.6+0x94c0b [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #05: 0x1491cc546c92 raise() from /lib64/libc.so.6+0x40c92 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #06: 0x1491cc52c49f abort() from /lib64/libc.so.6+0x2649f [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #07: 0x1491cc52d27e () from /lib64/libc.so.6+0x2727e [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #08: 0x1491cc5a43f5 () from /lib64/libc.so.6+0x9e3f5 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #09: 0x1491cc5a63c0 () from /lib64/libc.so.6+0xa03c0 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #10: 0x1491cc5a66da () from /lib64/libc.so.6+0xa06da [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #11: 0x1491cc5a8f3f free() from /lib64/libc.so.6+0xa2f3f [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #12: 0x435e11 () from /usr/sbin/smartdns+0x35e11 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #13: 0x435e75 () from /usr/sbin/smartdns+0x35e75 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #14: 0x435ea4 () from /usr/sbin/smartdns+0x35ea4 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #15: 0x416726 () from /usr/sbin/smartdns+0x16726 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #16: 0x416a10 () from /usr/sbin/smartdns+0x16a10 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #17: 0x417430 () from /usr/sbin/smartdns+0x17430 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #18: 0x41e2ef () from /usr/sbin/smartdns+0x1e2ef [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #19: 0x41c350 () from /usr/sbin/smartdns+0x1c350 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #20: 0x41c4cc () from /usr/sbin/smartdns+0x1c4cc [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #21: 0x41c554 () from /usr/sbin/smartdns+0x1c554 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #22: 0x420864 () from /usr/sbin/smartdns+0x20864 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #23: 0x420b53 () from /usr/sbin/smartdns+0x20b53 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #24: 0x40dbfa () from /usr/sbin/smartdns+0xdbfa [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #25: 0x410f60 () from /usr/sbin/smartdns+0x10f60 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #26: 0x413461 () from /usr/sbin/smartdns+0x13461 [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #27: 0x1491cc598eba () from /lib64/libc.so.6+0x92eba [2023-09-30 16:46:53,529][FATAL][ util.c:1455] #28: 0x1491cc623808 () from /lib64/libc.so.6+0x11d808

[micron10]

and one more :

[2023-09-30 18:48:21,408][ERROR][ nftset.c:591 ] nftset add failed, family:ip, table:raw, set:allows, error:File exists [2023-09-30 18:48:21,988][ERROR][ nftset.c:591 ] nftset add failed, family:ip, table:raw, set:allows, error:File exists [2023-09-30 18:48:21,997][ERROR][ nftset.c:591 ] nftset add failed, family:ip, table:raw, set:allows, error:File exists [2023-09-30 18:48:22,006][ERROR][ nftset.c:591 ] nftset add failed, family:ip, table:raw, set:allows, error:File exists [2023-09-30 18:48:22,340][ERROR][ nftset.c:591 ] nftset add failed, family:ip, table:raw, set:allows, error:File exists [2023-09-30 18:48:22,639][ERROR][ nftset.c:591 ] nftset add failed, family:ip, table:raw, set:allows, error:File exists

this may be come from : nftset-debug yes

[PikuZheng]

[2023-09-30 16:46:53,528][ERROR][ dns_cache.c:87 ] BUG: dns_cache is invalid.

this issue occurs if upgrading from 41.0.22 or earlier. delete cache file artificially may be helpful.

] nftset add failed, family:ip, table:raw, set:allows, error:File exists

this issue occurs when smartdns is not run as authorized user (like root) . but I'm not sure if you are

[micron10]

[2023-09-30 16:46:53,528][ERROR][ dns_cache.c:87 ] BUG: dns_cache is invalid.

this issue occurs if upgrading from 41.0.22 or earlier. delete cache file artificially may be helpful.

ok will test after remove.

] nftset add failed, family:ip, table:raw, set:allows, error:File exists

this issue occurs when smartdns is not run as authorized user (like root) . but I'm not sure if you are

run as user root.

service try to add same rull in nft and nft back file exists.

may be need to have check if ip is list in set to not try to add.

[micron10]

and one other:

to many lines with this:

[2023-09-30 20:43:40,315][ERROR][ dns_server.c:1908] update packet info failed. [2023-09-30 20:43:40,315][ERROR][ dns_server.c:1972] update packet ttl failed. [2023-09-30 20:43:40,315][ERROR][ dns_server.c:1908] update packet info failed. [2023-09-30 20:43:40,315][ERROR][ dns_server.c:1972] update packet ttl failed.

[PikuZheng]

nftables allows adding the same IP repeatedly without causing errors.

Are you compiling smartdns yourself? I suspect it has something to do with compilation

[micron10]

Hi Yes i compiling smartdns,

see this :

[2023-10-02 13:58:59,069][ERROR][ dns_server.c:7148] load cert /etc/smartdns/smartdns-cert.pem failed, error:8000000D:system library::Permission denied

this is how i compile :

make -j 60
CC=/build/x86_64-pc-linux-gnu/bin/x86_64-pc-linux-gnu-gcc
LD=/build/x86_64-pc-linux-gnu/bin/x86_64-pc-linux-gnu-ld
CFLAGS="-Og -ggdb3 -fno-omit-frame-pointer -DFEATURE_DWITH_NFTSET=1 -DDEBUG "
LDFLAGS="-L/lib -L/usr/lib -lmnl -liconv -lssl -lcrypto -lnftables -lreadline -lncurses -ltinfo -lnftnl -lxtables -lz -lgmp -ljansson -lnetfilter_conntrack -lnfnetlink"
DEBUG=1
-C /build/smartdns-master

[PikuZheng]

Perhaps your OS has advanced ACL controls that prevent the smartdns process from accessing some files. It's also possible that libopenssl is incorrect when compiling or running. The above information confuses me. I have no more ideas now.

[micron10]

find afgter set permision all is fine :

chmod 0700 -R /etc/smartdns/ chmod 0600 smartdns-* (cert key)

P.S. i try with build file from here and get same error , after fix permision now work.

Please for this if you have idea :

[2023-09-30 20:43:40,315][ERROR][ dns_server.c:1908] update packet info failed. [2023-09-30 20:43:40,315][ERROR][ dns_server.c:1972] update packet ttl failed. [2023-09-30 20:43:40,315][ERROR][ dns_server.c:1908] update packet info failed. [2023-09-30 20:43:40,315][ERROR][ dns_server.c:1972] update packet ttl failed.

[PikuZheng]

[2023-09-30 20:43:40,315][ERROR][ dns_server.c:1908] update packet info failed. [2023-09-30 20:43:40,315][ERROR][ dns_server.c:1972] update packet ttl failed. [2023-09-30 20:43:40,315][ERROR][ dns_server.c:1908] update packet info failed. [2023-09-30 20:43:40,315][ERROR][ dns_server.c:1972] update packet ttl failed.

Some DNS requests do not have these attributes. If smartdns works fine, please ignore it.

[micron10]

Thanks for now i will close if find any other will update you.

Thanks for support!

[micron10]

Hi @PikuZheng

please see this bug

[2023-12-29 22:20:13,358][FATAL][ smartdns.c:635 ] process exit with signal 11, code = 1, errno = 0, pid = 8, self = 396495, pc = 0x951aa0, addr = 0x8, build(Dec 29 2023 14:53:00 x86_64) [2023-12-29 22:20:13,359][FATAL][ util.c:1486] Stack: [2023-12-29 22:20:13,359][FATAL][ util.c:1498] #01: 0x439965 () from /usr/sbin/smartdns+0x39965 [2023-12-29 22:20:13,359][FATAL][ util.c:1498] #02: 0x4329ea () from /usr/sbin/smartdns+0x329ea [2023-12-29 22:20:13,359][FATAL][ util.c:1498] #03: 0x14ac2a484d30 () from /lib64/libc.so.6+0x40d30 [2023-12-29 22:20:13,359][FATAL][ util.c:1498] #04: 0x43f3b5 () from /usr/sbin/smartdns+0x3f3b5 [2023-12-29 22:20:13,359][FATAL][ util.c:1498] #05: 0x43f58a () from /usr/sbin/smartdns+0x3f58a [2023-12-29 22:20:13,360][FATAL][ util.c:1498] #06: 0x14ac2a4d6eba () from /lib64/libc.so.6+0x92eba [2023-12-29 22:20:13,360][FATAL][ util.c:1498] #07: 0x14ac2a561818 () from /lib64/libc.so.6+0x11d818

daemon run with -R

smartdns Last-Build-With-Kernel-6.4.13-85-g3d11a8f-dirty

[pymumu]

Upload the configuration file, as well as the frequency of problems and scenarios.

[micron10]

Hi

smartdns.conf user root

bind [::]:53 bind-tcp [::]:53 bind-tls [::]:853

server 8.8.8.8 server-tls 8.8.8.8 server 8.8.4.4 server-tls 8.8.4.4 server 1.1.1.1 server-tls 1.1.1.1 server 1.0.0.1 server-tls 1.0.0.1 #server 2001:4860:4860::8888 #server 2001:4860:4860::8844 #server 2606:4700::1111 #server-tls 2606:4700::1111 #server 2606:4700:4700::1001 #server-tls 2606:4700:4700::1001

cache-size 2621440 cache-persist yes cache-file /var/run/smartdns.cache cache-checkpoint-time 86400 prefetch-domain yes serve-expired yes serve-expired-ttl 259200 serve-expired-prefetch-time 21600

tcp-idle-time 120

force-qtype-SOA 65

rr-ttl 600

Enable IPV4, IPV6 dual stack IP optimization selection strategy

dualstack-ip-selection-threshold [num] (0~1000)

dualstack-ip-allow-force-AAAA [yes|no]

dualstack-ip-selection no

set log level

log-level: [level], level=off, fatal, error, warn, notice, info, debug

log-file: file path of log file.

log-console [yes|no]: output log to console.

log-size: size of each log file, support k,m,g

log-num: number of logs, 0 means disable log

log-level error

log-file /var/log/smartdns/smartdns.log log-size 20M log-num 2

enalbe DNS64 feature

dns64 [ip/subnet]

dns64 64:ff9b::/96

nftset-timeout yes

enable nftset debug, check nftset setting result, output log when error.

nftset-debug [yes|no]

nftset-debug yes

conf-file msallow.conf

scenario , on firewall have rull that block port 53 from internet request. in side on machine have 7.5k users on pppoe and use dns as primary. all is fine but only i see from time to time child dead and rerun.

396494 smartdns Fri Dec 29 20:24:34 2023 02:24:49 00:00:00 /usr/sbin/smartdns -R 462881 smartdns Fri Dec 29 22:20:14 2023 29:09 00:23:25 /usr/sbin/smartdns -R

i try to build service with this :

make -j30
DEBUG=1
CFLAGS="-Og -ggdb3 -fno-omit-frame-pointer -I/include -I/usr/include -DFEATURE_DWITH_NFTSET=1 -DDEBUG "

but in smardns.log i dont see full debug

[pymumu]

If possible, use "ulimit -c unlimited" enable coredump. When starting smartdns, add the -S parameter to generate coredump file when crash and upload the coredump file and executable file.

And better tell what the commit id of the corresponding code is. It would be best if you can debug it with gdb

Or you can run addr2line -e smartdns [address] to show line number in the source code where crash occured.

Build command

make DEBUG=1 -j

[micron10]

Hi @pymumu Happy New Year !

when have time download core dump size is 700mb:

https://fileport.io/EZUw7vNdw7Xe

P.S.

In log file have only one line in this period:

[2023-12-31 17:51:45,356][ERROR][ dns_cache.c:96 ] BUG: dns_cache is invalid.

before run build with latest master i remove old cache file .

One more this is gdb read of core dump :

gdb /usr/sbin/smartdns core-smartdns-981335

https://pastebin.com/raw/VDD5Wz3b

[pymumu]

Try the latest code please.

I have modified a few possible places, but I am not sure whether it can solve the problem.

[micron10]

I think all is fine now .

Now build latest master and get this warrning please see:

dns_conf.c: In function '_get_domain': dns_conf.c:346:30: warning: implicit declaration of function 'utf8_to_punycode' [-Wimplicit-function-declaration] 346 | domain_len = utf8_to_punycode(begin, len, domain, domain_len); | ^~~~~~~~~~~~~~~~

[micron10]

And one more is it possible this :

src/dns_server.c

if (type == SOCK_STREAM) { if (listen(fd, 16) != 0) { tlog(TLOG_ERROR, "listen failed.\n"); goto errout; } }

listen(fd, 16) - to incress

in dmesg i get this error :

[Tue Jan 16 13:57:28 2024] TCP: request_sock_TCP: Possible SYN flooding on port [::]:53. Dropping request. [Tue Jan 16 14:22:57 2024] TCP: request_sock_TCP: Possible SYN flooding on port [::]:53. Dropping request. [Tue Jan 16 14:58:38 2024] TCP: request_sock_TCP: Possible SYN flooding on port [::]:53. Dropping request.

and here : https://access.redhat.com/solutions/30453

Increase application socket listen backlog - Search this part recommend to incress.

[pymumu]

implicit-function-declaration There is no problem when I compile. What version of gcc?

[micron10]

implicit-function-declaration There is no problem when I compile. What version of gcc?

gcc 13.2.0 glibc 2.38

[micron10]

And one more is it possible this :

src/dns_server.c

if (type == SOCK_STREAM) { if (listen(fd, 16) != 0) { tlog(TLOG_ERROR, "listen failed.\n"); goto errout; } }

listen(fd, 16) - to incress

in dmesg i get this error :

[Tue Jan 16 13:57:28 2024] TCP: request_sock_TCP: Possible SYN flooding on port [::]:53. Dropping request. [Tue Jan 16 14:22:57 2024] TCP: request_sock_TCP: Possible SYN flooding on port [::]:53. Dropping request. [Tue Jan 16 14:58:38 2024] TCP: request_sock_TCP: Possible SYN flooding on port [::]:53. Dropping request.

and here : https://access.redhat.com/solutions/30453

Increase application socket listen backlog - Search this part recommend to incress.

thanks for fix :

https://github.com/pymumu/smartdns/commit/5fec5261f9c54eb3061dfb3c658169c2ab9cbae8

[micron10]

Hi @pymumu

one strange situation

when start service all is fine work on 2-9% load..

but in time of work after 1 hour or more start use 100% and more.

how to catch why ?

[pymumu]

top，gdb backtrace，log，audit-log config file

[micron10]

Config :

user root

bind [::]:53 bind-tcp [::]:53 bind-tls [::]:853

server 8.8.8.8 server-tls 8.8.8.8 server 8.8.4.4 server-tls 8.8.4.4 server 1.1.1.1 server-tls 1.1.1.1 server 1.0.0.1 server-tls 1.0.0.1

cache-size 2621440 cache-persist yes cache-file /var/run/smartdns.cache cache-checkpoint-time 86400 prefetch-domain yes serve-expired yes serve-expired-ttl 259200 serve-expired-prefetch-time 21600

tcp-idle-time 120

force-qtype-SOA 65

rr-ttl 600

Enable IPV4, IPV6 dual stack IP optimization selection strategy

dualstack-ip-selection-threshold [num] (0~1000)

dualstack-ip-allow-force-AAAA [yes|no]

dualstack-ip-selection no

set log level

log-level: [level], level=off, fatal, error, warn, notice, info, debug

log-file: file path of log file.

log-console [yes|no]: output log to console.

log-size: size of each log file, support k,m,g

log-num: number of logs, 0 means disable log

log-level error

log-file /var/log/smartdns/smartdns.log log-size 20M log-num 2

enalbe DNS64 feature

dns64 [ip/subnet]

dns64 64:ff9b::/96

nftset-timeout yes

enable nftset debug, check nftset setting result, output log when error.

nftset-debug [yes|no]

nftset-debug yes

Log :

[2024-01-23 08:54:12,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 08:54:12,108][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 08:54:12,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 08:54:12,108][ERROR][ dns_server.c:4055] update packet ttl failed. [2024-01-23 08:54:19,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 08:54:19,108][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 08:54:19,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 08:54:19,108][ERROR][ dns_server.c:4055] update packet ttl failed. [2024-01-23 08:54:26,308][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 08:54:26,308][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 08:54:26,308][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 08:54:26,308][ERROR][ dns_server.c:4055] update packet ttl failed. [2024-01-23 09:48:16,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 09:48:16,108][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 09:48:16,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 09:48:16,108][ERROR][ dns_server.c:4055] update packet ttl failed. [2024-01-23 09:48:18,208][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 09:48:18,208][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 09:48:23,208][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 09:48:23,208][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 09:48:26,308][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 09:48:26,308][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 09:48:26,308][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 09:48:26,308][ERROR][ dns_server.c:4055] update packet ttl failed. [2024-01-23 09:48:26,308][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 09:48:26,308][ERROR][ dns_server.c:4055] update packet ttl failed. [2024-01-23 09:48:26,308][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 09:48:26,308][ERROR][ dns_server.c:4055] update packet ttl failed. [2024-01-23 11:02:57,708][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 11:02:57,708][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 11:02:57,908][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 11:02:57,908][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:07:48,308][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:07:48,308][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:07:53,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:07:53,108][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:10:27,608][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:10:27,608][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:12:50,008][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:12:50,009][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:12:53,408][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:12:53,408][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:12:58,308][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:12:58,308][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:12:58,308][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:12:58,308][ERROR][ dns_server.c:4055] update packet ttl failed. [2024-01-23 12:15:30,008][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:15:30,008][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:15:30,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:15:30,108][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:19:01,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:19:01,108][ERROR][ dns_server.c:2227] update packet ttl failed. [2024-01-23 12:19:06,108][ERROR][ dns_server.c:2163] update packet info failed. [2024-01-23 12:19:06,108][ERROR][ dns_server.c:2227] update packet ttl failed.

top :

top - 18:56:19 up 54 days, 1:03, 1 user, load average: 0.96, 0.69, 0.55 Tasks: 1 total, 1 running, 0 sleeping, 0 stopped, 0 zombie %Cpu0 : 0.0 us, 0.0 sy, 0.0 ni, 74.8 id, 0.0 wa, 0.0 hi, 25.2 si, 0.0 st %Cpu1 : 0.0 us, 0.0 sy, 0.0 ni, 74.0 id, 0.0 wa, 0.0 hi, 26.0 si, 0.0 st %Cpu2 : 0.0 us, 0.7 sy, 0.7 ni, 74.5 id, 0.0 wa, 0.0 hi, 24.2 si, 0.0 st %Cpu3 : 0.7 us, 0.7 sy, 0.0 ni, 77.7 id, 0.0 wa, 0.0 hi, 20.9 si, 0.0 st %Cpu4 : 18.1 us, 24.8 sy, 0.0 ni, 30.9 id, 0.0 wa, 0.0 hi, 26.2 si, 0.0 st %Cpu5 : 0.7 us, 0.7 sy, 0.0 ni, 76.4 id, 0.0 wa, 0.0 hi, 22.3 si, 0.0 st %Cpu6 : 0.0 us, 0.0 sy, 0.0 ni, 75.0 id, 0.0 wa, 0.0 hi, 25.0 si, 0.0 st %Cpu7 : 0.0 us, 0.0 sy, 0.0 ni, 80.8 id, 0.0 wa, 0.0 hi, 19.2 si, 0.0 st %Cpu8 : 0.0 us, 0.0 sy, 0.0 ni, 80.0 id, 0.0 wa, 0.0 hi, 20.0 si, 0.0 st %Cpu9 : 0.0 us, 0.0 sy, 0.0 ni, 76.6 id, 0.0 wa, 0.0 hi, 23.4 si, 0.0 st %Cpu10 : 0.0 us, 0.0 sy, 0.0 ni, 80.7 id, 0.0 wa, 0.0 hi, 19.3 si, 0.0 st %Cpu11 : 0.7 us, 0.7 sy, 0.0 ni, 73.5 id, 0.0 wa, 0.0 hi, 25.2 si, 0.0 st %Cpu12 : 6.1 us, 6.8 sy, 0.0 ni, 59.5 id, 0.0 wa, 0.0 hi, 27.7 si, 0.0 st %Cpu13 : 0.0 us, 0.0 sy, 0.0 ni, 73.8 id, 0.0 wa, 0.0 hi, 26.2 si, 0.0 st %Cpu14 : 0.0 us, 0.0 sy, 0.0 ni, 78.1 id, 0.0 wa, 0.0 hi, 21.9 si, 0.0 st %Cpu15 : 1.4 us, 0.0 sy, 0.0 ni, 73.6 id, 0.0 wa, 0.0 hi, 25.0 si, 0.0 st %Cpu16 : 0.0 us, 0.0 sy, 0.0 ni, 74.5 id, 0.0 wa, 0.0 hi, 25.5 si, 0.0 st %Cpu17 : 0.0 us, 0.7 sy, 0.0 ni, 72.0 id, 0.0 wa, 0.0 hi, 27.3 si, 0.0 st %Cpu18 : 0.7 us, 0.7 sy, 0.0 ni, 70.3 id, 0.0 wa, 0.0 hi, 28.4 si, 0.0 st %Cpu19 : 0.7 us, 0.7 sy, 0.0 ni, 78.6 id, 0.0 wa, 0.0 hi, 20.0 si, 0.0 st %Cpu20 : 0.7 us, 0.0 sy, 0.0 ni, 75.3 id, 0.0 wa, 0.0 hi, 24.0 si, 0.0 st %Cpu21 : 0.0 us, 0.0 sy, 0.0 ni, 78.4 id, 0.0 wa, 0.0 hi, 21.6 si, 0.0 st %Cpu22 : 5.4 us, 7.5 sy, 0.0 ni, 66.7 id, 0.0 wa, 0.0 hi, 20.4 si, 0.0 st %Cpu23 : 0.7 us, 0.0 sy, 0.0 ni, 76.9 id, 0.0 wa, 0.0 hi, 22.4 si, 0.0 st %Cpu24 : 0.7 us, 0.7 sy, 0.0 ni, 76.7 id, 0.0 wa, 0.0 hi, 21.9 si, 0.0 st %Cpu25 : 0.0 us, 0.7 sy, 0.0 ni, 74.8 id, 0.0 wa, 0.0 hi, 24.5 si, 0.0 st %Cpu26 : 0.0 us, 0.7 sy, 0.0 ni, 81.2 id, 0.0 wa, 0.0 hi, 18.1 si, 0.0 st %Cpu27 : 0.0 us, 0.0 sy, 0.0 ni, 78.8 id, 0.0 wa, 0.0 hi, 21.2 si, 0.0 st %Cpu28 : 0.0 us, 0.7 sy, 0.7 ni, 72.0 id, 0.0 wa, 0.0 hi, 26.7 si, 0.0 st %Cpu29 : 0.0 us, 0.0 sy, 0.0 ni, 73.5 id, 0.0 wa, 0.0 hi, 26.5 si, 0.0 st %Cpu30 : 0.0 us, 0.0 sy, 0.0 ni, 77.7 id, 0.0 wa, 0.0 hi, 22.3 si, 0.0 st %Cpu31 : 0.0 us, 0.0 sy, 0.0 ni, 81.0 id, 0.0 wa, 0.0 hi, 19.0 si, 0.0 st MiB Mem : 31927.0 total, 24657.1 free, 5024.1 used, 3475.4 buff/cache MiB Swap: 0.0 total, 0.0 free, 0.0 used. 26902.9 avail Mem

PID USER      PR  NI    VIRT    RES  %CPU  %MEM     TIME+ S COMMAND

1883577 root 20 0 292.1m 25.5m 94.7 0.1 1:04.95 R /usr/sbin/smartdns -R -S

autdit-log dont have.

for gdb backtrace will need your help how to get.

service not crash only start use more cpu .

ver : smartdns 1.20240116-1435

[PikuZheng]

can you check the file size /var/run/smartdns.cache

[micron10]

737M /var/run/smartdns.cache

[pymumu]

1. Run top -H -p [pid] to find which thread of smartdns is occupying too much CPU.

2. After turning on the audit log, check the audit log status

   audit-enable yes
   

3. Build smartdns with debug info.

    export DEBUG=1
    make all -j

1. When the program has too many CPUs, run gdb attach [pid], after execution

   t a a bt
   

[pymumu]

About the log: update packet ttl failed Please modify the code, and reproduce the issue, and attach the file: /tmp/dns_packet

static int _dns_request_update_id_ttl(struct dns_server_post_context *context)
{
	int ttl = context->reply_ttl;
	struct dns_request *request = context->request;

	if (request->conf->dns_rr_ttl_reply_max > 0) {
		if (request->ip_ttl > request->conf->dns_rr_ttl_reply_max && ttl == 0) {
			ttl = request->ip_ttl;
		}

		if (ttl > request->conf->dns_rr_ttl_reply_max) {
			ttl = request->conf->dns_rr_ttl_reply_max;
		}

		if (ttl == 0) {
			ttl = request->conf->dns_rr_ttl_reply_max;
		}
	}

	if (ttl == 0) {
		ttl = request->ip_ttl;
		if (ttl == 0) {
			ttl = _dns_server_get_conf_ttl(request, ttl);
		}
	}

	struct dns_update_param param;
	param.id = request->id;
	param.cname_ttl = ttl;
	param.ip_ttl = ttl;
	if (dns_packet_update(context->inpacket, context->inpacket_len, &param) != 0) {
		int fd = open("/tmp/dns_packet", O_WRONLY | O_CREAT | O_TRUNC, 0644);
		if (fd > 0) {
			write(fd, context->inpacket, context->inpacket_len);
			close(fd);
		}
		tlog(TLOG_ERROR, "update packet info failed.");
		return -1;
	}

	return 0;
}

[micron10]

Build with this .

see this is gdb from now :

Thread 1 (LWP 1905573 "smartdns"): #0 0x00001493d47cf597 in recvmsg () from /lib64/libc.so.6 #1 0x00001493d47e71ae in ?? () from /lib64/libc.so.6 #2 0x00001493d47e7463 in ?? () from /lib64/libc.so.6 #3 0x00001493d47e8140 in getifaddrs () from /lib64/libc.so.6 #4 0x0000000000425a6d in _dns_server_process_local_ptr (request=request@entry=0x590e580) at dns_server.c:4589 #5 0x000000000042a9b8 in _dns_server_process_ptr (request=request@entry=0x590e580) at dns_server.c:4711 #6 0x000000000042a9dd in _dns_server_process_ptr_query (request=request@entry=0x590e580) at dns_server.c:5959 #7 0x000000000042c920 in _dns_server_do_query (request=request@entry=0x590e580, skip_notify_event=skip_notify_event@entry=0) at dns_server.c:6363 #8 0x000000000042ce39 in _dns_server_prefetch_request (domain=domain@entry=0x58f9620 "38.0.168.192.in-addr.arpa", qtype=DNS_T_PTR, server_query_option=server_query_option@entry=0x7fff216adc40, prefetch_flag=) at dns_server.c:6631 #9 0x000000000042d16b in _dns_server_process_cache (request=request@entry=0x58f95b0) at dns_server.c:5792 #10 0x000000000042c95a in _dns_server_do_query (request=request@entry=0x58f95b0, skip_notify_event=skip_notify_event@entry=1) at dns_server.c:6357 #11 0x000000000042d6bd in _dns_server_recv (conn=conn@entry=0x22b4ab0, inpacket=inpacket@entry=0x7fff216b13f0 "\037\033\001", inpacket_len=inpacket_len@entry=43, local=local@entry=0x7fff216b12f0, local_len=16, from=from@entry=0x7fff216b1370, from_len=28) at dns_server.c:6574 #12 0x000000000042da8e in _dns_server_process_udp_one (udpconn=udpconn@entry=0x22b4ab0, event=event@entry=0x7fff216b35b0, now=now@entry=4673745560) at dns_server.c:6726 #13 0x000000000042dacd in _dns_server_process_udp (udpconn=udpconn@entry=0x22b4ab0, event=event@entry=0x7fff216b35b0, now=now@entry=4673745560) at dns_server.c:6733 #14 0x000000000042e194 in _dns_server_process (conn=0x22b4ab0, event=event@entry=0x7fff216b35b0, now=now@entry=4673745560) at dns_server.c:7499 #15 0x000000000042e96b in dns_server_run () at dns_server.c:7936 #16 0x0000000000436c67 in _smartdns_run () at smartdns.c:670 #17 0x000000000043728a in main (argc=4, argv=) at smartdns.c:1139

this is top :

top - 20:08:59 up 54 days, 2:16, 1 user, load average: 0.55, 0.61, 0.57 Threads: 6 total, 1 running, 5 sleeping, 0 stopped, 0 zombie %Cpu0 : 0.0 us, 0.0 sy, 0.0 ni, 72.8 id, 0.0 wa, 0.0 hi, 27.2 si, 0.0 st %Cpu1 : 0.0 us, 0.0 sy, 0.0 ni, 71.3 id, 0.0 wa, 0.0 hi, 28.7 si, 0.0 st %Cpu2 : 0.0 us, 0.0 sy, 0.0 ni, 76.4 id, 0.0 wa, 0.0 hi, 23.6 si, 0.0 st %Cpu3 : 0.0 us, 0.7 sy, 0.0 ni, 69.1 id, 0.0 wa, 0.0 hi, 30.2 si, 0.0 st %Cpu4 : 0.0 us, 0.7 sy, 0.0 ni, 75.5 id, 0.0 wa, 0.0 hi, 23.8 si, 0.0 st %Cpu5 : 2.7 us, 0.7 sy, 0.0 ni, 66.7 id, 0.0 wa, 0.0 hi, 30.0 si, 0.0 st %Cpu6 : 0.0 us, 0.0 sy, 0.0 ni, 71.8 id, 0.0 wa, 0.0 hi, 28.2 si, 0.0 st %Cpu7 : 0.0 us, 1.4 sy, 0.7 ni, 76.4 id, 0.0 wa, 0.0 hi, 21.6 si, 0.0 st %Cpu8 : 0.0 us, 0.0 sy, 0.0 ni, 73.5 id, 0.0 wa, 0.0 hi, 26.5 si, 0.0 st %Cpu9 : 0.0 us, 0.0 sy, 0.0 ni, 69.5 id, 0.0 wa, 0.0 hi, 30.5 si, 0.0 st %Cpu10 : 0.0 us, 0.0 sy, 0.0 ni, 73.0 id, 0.0 wa, 0.0 hi, 27.0 si, 0.0 st %Cpu11 : 0.0 us, 0.0 sy, 0.0 ni, 77.6 id, 0.0 wa, 0.0 hi, 22.4 si, 0.0 st %Cpu12 : 0.0 us, 0.0 sy, 0.0 ni, 73.6 id, 0.0 wa, 0.0 hi, 26.4 si, 0.0 st %Cpu13 : 0.0 us, 0.0 sy, 0.0 ni, 73.8 id, 0.0 wa, 0.0 hi, 26.2 si, 0.0 st %Cpu14 : 30.0 us, 41.3 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 28.7 si, 0.0 st %Cpu15 : 0.7 us, 0.7 sy, 0.0 ni, 72.1 id, 0.0 wa, 0.0 hi, 26.5 si, 0.0 st %Cpu16 : 0.0 us, 0.0 sy, 0.0 ni, 74.3 id, 0.0 wa, 0.0 hi, 25.7 si, 0.0 st %Cpu17 : 1.4 us, 0.7 sy, 0.0 ni, 73.0 id, 0.0 wa, 0.0 hi, 25.0 si, 0.0 st %Cpu18 : 0.0 us, 0.0 sy, 0.0 ni, 71.2 id, 0.0 wa, 0.0 hi, 28.8 si, 0.0 st %Cpu19 : 0.0 us, 0.0 sy, 0.0 ni, 73.6 id, 0.0 wa, 0.0 hi, 26.4 si, 0.0 st %Cpu20 : 0.0 us, 0.0 sy, 0.0 ni, 80.3 id, 0.0 wa, 0.0 hi, 19.7 si, 0.0 st %Cpu21 : 0.7 us, 1.4 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 24.7 si, 0.0 st %Cpu22 : 2.0 us, 1.3 sy, 0.0 ni, 73.3 id, 0.0 wa, 0.0 hi, 23.3 si, 0.0 st %Cpu23 : 0.7 us, 0.0 sy, 0.0 ni, 74.7 id, 0.0 wa, 0.0 hi, 24.7 si, 0.0 st %Cpu24 : 0.0 us, 0.7 sy, 0.0 ni, 76.2 id, 0.0 wa, 0.0 hi, 23.1 si, 0.0 st %Cpu25 : 0.0 us, 0.0 sy, 0.0 ni, 68.7 id, 0.0 wa, 0.0 hi, 31.3 si, 0.0 st %Cpu26 : 0.0 us, 0.0 sy, 0.0 ni, 68.9 id, 0.0 wa, 0.0 hi, 31.1 si, 0.0 st %Cpu27 : 0.0 us, 0.0 sy, 0.0 ni, 75.5 id, 0.0 wa, 0.0 hi, 24.5 si, 0.0 st %Cpu28 : 0.0 us, 0.0 sy, 0.0 ni, 77.2 id, 0.0 wa, 0.0 hi, 22.8 si, 0.0 st %Cpu29 : 0.0 us, 0.0 sy, 0.0 ni, 72.7 id, 0.0 wa, 0.0 hi, 27.3 si, 0.0 st %Cpu30 : 0.0 us, 0.0 sy, 0.0 ni, 64.6 id, 0.0 wa, 0.0 hi, 35.4 si, 0.0 st %Cpu31 : 0.7 us, 0.0 sy, 0.0 ni, 73.0 id, 0.0 wa, 0.0 hi, 26.4 si, 0.0 st MiB Mem : 31927.0 total, 24551.6 free, 5138.7 used, 3500.4 buff/cache MiB Swap: 0.0 total, 0.0 free, 0.0 used. 26788.3 avail Mem

PID USER      PR  NI    VIRT    RES  %CPU  %MEM     TIME+ S COMMAND

1905573 root 20 0 424.6m 110.0m 99.9 0.3 10:50.35 R /usr/sbin/smartdns -R -S 1905578 root 20 0 424.6m 110.0m 2.0 0.3 0:29.10 S /usr/sbin/smartdns -R -S 1905574 root 20 0 424.6m 110.0m 1.3 0.3 0:22.39 S /usr/sbin/smartdns -R -S 1905576 root 15 -5 424.6m 110.0m 0.7 0.3 0:07.13 S /usr/sbin/smartdns -R -S 1905577 root 20 0 424.6m 110.0m 0.7 0.3 0:05.98 S /usr/sbin/smartdns -R -S 1905575 root 20 0 424.6m 110.0m 0.0 0.3 0:02.90 S /usr/sbin/smartdns -R -S

now will replease with rebuilded service .

P.S.

see this in ps fax :

1905572 ? S 0:00 /usr/sbin/smartdns -R -S 1905573 ? Rl 0:46 _ /usr/sbin/smartdns -R -S 1906307 ? Z 0:00 _ [gzip]

this is in log folder :

-rw-r----- 1 root root 472946 Jan 23 19:39 /var/log/smartdns/smartdns-audit.log -r--r----- 1 root root 207315 Jan 23 19:39 /var/log/smartdns/smartdns-audit.log-20240123-193942.gz -r--r----- 1 root root 202394 Jan 23 19:39 /var/log/smartdns/smartdns-audit.log-20240123-193947.gz -rw-r----- 1 root root 237641 Jan 23 19:39 /var/log/smartdns/smartdns.log -r--r----- 1 root root 2811813 Dec 2 11:02 /var/log/smartdns/smartdns.log-20231202-110248.gz -r--r----- 1 root root 2436360 Jan 16 14:19 /var/log/smartdns/smartdns.log-20240116-141942.gz

smartdns-audit.log-20240123-201050.gz

[pymumu]

Try adding this configuration to disable PTR query.

force-qtype-SOA 12