vpp icon indicating copy to clipboard operation
vpp copied to clipboard

perfmon: asan reports global-buffer-overflow on Intel Xeons

Open vrpolakatcisco opened this issue 3 months ago • 0 comments

When I attempt to replicate some issue in CSIT using build with address sanitizer (asan), it is currently affected by two issues. This is the older one, already present in VPP 24.10-release (older releases do not compile on Ubuntu 24.04). When I edit CSIT code to not enable perfmon, VPP starts normally. On Atoms (such as 3n-snr), VPP starts normally. Recently I finally managed to edit CSIT code so it logs the asan output (visible here [0] if you scroll down enough):

==1032329==ERROR: AddressSanitizer: global-buffer-overflow on address 0x7ffff43f1f78 at pc 0x7ffff43c92b7 bp 0x7fff2b200010 sp 0x7fff2b200008
READ of size 8 at 0x7ffff43f1f78 thread T0
==1032329==WARNING: invalid path to external symbolizer!
==1032329==WARNING: Failed to use and restart external symbolizer!
    #0 0x7ffff43c92b6  (/usr/lib/x86_64-linux-gnu/vpp_plugins/perfmon_plugin.so+0x2d2b6) (BuildId: 2ea45dd9553a60c55161b83e2144485424783557)
    #1 0x7ffff43c7f36  (/usr/lib/x86_64-linux-gnu/vpp_plugins/perfmon_plugin.so+0x2bf36) (BuildId: 2ea45dd9553a60c55161b83e2144485424783557)
    #2 0x7ffff43c662c  (/usr/lib/x86_64-linux-gnu/vpp_plugins/perfmon_plugin.so+0x2a62c) (BuildId: 2ea45dd9553a60c55161b83e2144485424783557)
    #3 0x7ffff43c0bfe  (/usr/lib/x86_64-linux-gnu/vpp_plugins/perfmon_plugin.so+0x24bfe) (BuildId: 2ea45dd9553a60c55161b83e2144485424783557)
    #4 0x7ffff529ddb4  (/lib/x86_64-linux-gnu/libvlib.so.24.10.0+0xaadb4) (BuildId: 424aaabf986e074d2e5dc64d710fd4c73498ed85)
    #5 0x7ffff529da0e  (/lib/x86_64-linux-gnu/libvlib.so.24.10.0+0xaaa0e) (BuildId: 424aaabf986e074d2e5dc64d710fd4c73498ed85)
    #6 0x7ffff529dec9  (/lib/x86_64-linux-gnu/libvlib.so.24.10.0+0xaaec9) (BuildId: 424aaabf986e074d2e5dc64d710fd4c73498ed85)
    #7 0x7ffff52beafd  (/lib/x86_64-linux-gnu/libvlib.so.24.10.0+0xcbafd) (BuildId: 424aaabf986e074d2e5dc64d710fd4c73498ed85)
    #8 0x7ffff539bc35  (/lib/x86_64-linux-gnu/libvlib.so.24.10.0+0x1a8c35) (BuildId: 424aaabf986e074d2e5dc64d710fd4c73498ed85)
    #9 0x7ffff5010567  (/lib/x86_64-linux-gnu/libvppinfra.so.24.10.0+0xb4567) (BuildId: 4876c4fa2c3aa331cf11749288aef5080220d5cd)

0x7ffff43f1f78 is located 40 bytes before global variable '__perfmon_bundle_intel_core_backend_bound_core' defined in '/w/workspace/vpp-csit-verify-perf-master-ubuntu2404-x86_64-2n-emr/src/plugins/perfmon/intel/bundle/backend_bound_core.c' (0x7ffff43f1fa0) of size 184
0x7ffff43f1f78 is located 0 bytes after global variable '.compoundliteral' defined in '/w/workspace/vpp-csit-verify-perf-master-ubuntu2404-x86_64-2n-emr/src/plugins/perfmon/intel/uncore.c' (0x7ffff43f1f40) of size 56
SUMMARY: AddressSanitizer: global-buffer-overflow (/usr/lib/x86_64-linux-gnu/vpp_plugins/perfmon_plugin.so+0x2d2b6) (BuildId: 2ea45dd9553a60c55161b83e2144485424783557) 
Shadow bytes around the buggy address:
  0x7ffff43f1c80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7ffff43f1d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7ffff43f1d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7ffff43f1e00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7ffff43f1e80: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
=>0x7ffff43f1f00: 00 f9 f9 f9 00 00 f9 f9 00 00 00 00 00 00 00[f9]
  0x7ffff43f1f80: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
  0x7ffff43f2000: 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9 f9 f9
  0x7ffff43f2080: 00 00 f9 f9 00 00 00 00 00 00 00 00 f9 f9 f9 f9
  0x7ffff43f2100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x7ffff43f2180: 00 00 00 00 00 00 00 f9 f9 f9 f9 f9 00 00 f9 f9
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==1032329==ABORTING

Not sure whether the symbolizer warnings are a separate issue or just me not using asan properly.

[0] https://logs.fd.io/vex-yul-rot-jenkins-1/vpp-csit-verify-perf-master-ubuntu2404-x86_64-2n-emr/11/csit_current/0/log.html.gz#s1-s1-s1-s1-s1-t1-k2-k5-k1-k1-k1-k1

vrpolakatcisco avatar Sep 03 '25 12:09 vrpolakatcisco