HANG (11.3.0) sample libcallstack.so does not work on AArch64 core

[kingjane]

This command does not output the call stack and must be terminated manually using Ctrl+C on openEuler 22.03 (LTS-SP4) running on a Kunpeng-920 core (AArch64). Adding the -debug option produces the output shown at this link, with the main error message being: <Application /usr/bin/ls (3706039). Internal Error: DynamoRIO debug check failure: /home/runner/work/dynamorio/dynamorio/core/unix/memcache.c:225 ALIGNED(start, PAGE_SIZE)

Versions

• What version of DynamoRIO are you using?
  • 11.3.0
• Does the latest build from https://github.com/DynamoRIO/dynamorio/releases solve the problem?
  • 11.9.0 does not solve the problem.
• What operating system version are you running on? ("Windows 10" is not sufficient: give the release number.)
  • openEuler 22.03 (LTS-SP4) running on a Kunpeng-920 core (AArch64)
• Is your application 32-bit or 64-bit?
  • 64-bit
• libc version
  • ldd (GNU libc) 2.34

[edeiana]

Can't reproduce on our internal aarch64 Ubuntu 22.04 LTS machine:

$ ./bin64/drrun -c samples/bin64/libcallstack.so -- ls
wrapping /usr/lib/aarch64-linux-gnu/libc.so.6!malloc
malloc called from:
  libc.so.6!<unknown>
  libc.so.6!<unknown>
  libselinux.so.1!<unknown>
  libselinux.so.1!<unknown>
  ld-linux-aarch64.so.1!<unknown>
  ld-linux-aarch64.so.1!<unknown>
  ld-linux-aarch64.so.1!<unknown>
  ld-linux-aarch64.so.1!<unknown>
[...]
malloc called from:
  libc.so.6!<unknown>
  libc.so.6!<unknown>
  libc.so.6!<unknown>
  libc.so.6!<unknown>
  libc.so.6!<unknown>
  ls!<unknown>
  libc.so.6!<unknown>
  libc.so.6!<unknown>
  ls!<unknown>
[...]
ACKNOWLEDGEMENTS  bin64  cmake	docs  docs_embed  ext  include	lib64  License.txt  logs  README  samples  tools

$ echo $?
0

[kingjane]

@edeiana That's a bit strange. Which specific core are you using?

I tested it on another machine running Rocky Linux 9.4 (Blue Onyx) with a Neoverse-N1 core (AArch64). The program also hangs after outputting several lines of the call stack, as shown in the picture above. However, when I add the -debug option, it runs normally. The output is in this link.

[abhinav92003]

I would suggest getting more information for debugging:

• For the original case "openEuler 22.03 (LTS-SP4) running on a Kunpeng-920 core (AArch64)" where a debug assert is hit: try running with logging enabled (https://dynamorio.org/page_logging.html) to see if any interesting information shows up before hitting the assert.
• Also, a symbolized callstack for the assert crash would be useful: https://dynamorio.org/page_debugging.htm.
• What happens if you run plain DR without any client?
• Can you get the callstack from when the non-debug build execution is presumably stuck, using gdb perhaps?

[abhinav92003]

Re https://github.com/DynamoRIO/dynamorio/issues/7337#issuecomment-2705935789: @kingjane Could you please edit your comment to shorten the example output (for repeated text, just replace with ...; or else, use https://gist.github.com/). This would avoid having to scroll all the way down on this page.

[kingjane]

@abhinav92003 Thank you for your suggestion. I have revised the comment.

[kingjane]

@abhinav92003 I reran the program with loglevel=2/3 and checked the log files, finding that the core information is identical to the output with the -debug option.

Since I'm new to DynamoRIO, many of the source files in dynamorio/core referenced in the log are unfamiliar to me. Debugging within the framework is somewhat challenging.

In addition, I tried building DynamoRIO from source, but the situation remains the same.

[kingjane]

@abhinav92003

• What happens if you run plain DR without any client? In the case of "openEuler 22.03 (LTS-SP4) running on a Kunpeng-920 core (AArch64)", the command ./bin64/drrun -- ls runs normally. However, running ./bin64/drrun -debug -- ls produces the same message as when using the "-c libcallstack.so" option. That means the error "<Application /usr/bin/ls (3706039). Internal Error: DynamoRIO debug check failure: /home/runner/work/dynamorio/dynamorio/core/unix/memcache.c:225 ALIGNED(start, PAGE_SIZE)" has nothing to do with libcallstack.so. Now, I need to determine the cause of this error and understand why libcallstack is not working.