codechecker icon indicating copy to clipboard operation
codechecker copied to clipboard

Published 20 hours ago • verified publisher icon Ericsson

Compilation database containing assembly tools considered invalid

Open gjasny opened this issue 3 years ago • 2 comments

Hello,

first, thanks for all your work to add nice and accessible tooling around clang and clang-tidy.

Describe the bug I have a CMake project that contains NASM/YASM assembly files besides a lot of C/C++. Due to the assembly part the compilation database is rejected by codechecker analyze.

CodeChecker version 6.19.1 (snap on Ubuntu 22.04)

To Reproduce

# install nasm beforehand
cmake_minimum_required(VERSION 3.15)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
project(testcase CXX ASM_NASM)
add_executable(testcase main.cpp assembly.asm)

the compilation database now contains something like that:

[
{
  "directory": "/home/gregorj/Git/Experimental/codechecker-nasm/_build",
  "command": "/usr/bin/c++ -o CMakeFiles/testcase.dir/main.cpp.o -c /home/gregorj/Git/Experimental/codechecker-nasm/main.cpp",
  "file": "/home/gregorj/Git/Experimental/codechecker-nasm/main.cpp"
},
{
  "directory": "/home/gregorj/Git/Experimental/codechecker-nasm/_build",
  "command": "/usr/bin/nasm -f elf64 -o CMakeFiles/testcase.dir/assembly.asm.o /home/gregorj/Git/Experimental/codechecker-nasm/assembly.asm",
  "file": "/home/gregorj/Git/Experimental/codechecker-nasm/assembly.asm"
}
]

If I run codechecker analyze on that I get the following error:

$ codechecker analyze _build/compile_commands.json  --output _reports --verbose debug
[DEBUG][2022-05-07 16:18:26] {system} [1630620] <140290582947648> - analyze.py:971 main() - args: Namespace(analyzer_config=['clang-tidy:HeaderFilterRegex=.*'], compile_uniqueing='none', config_file=None, func=<function main at 0x7f97ef123b70>, func_process_config_file=<function process_config_file at 0x7f97ef1236a8>, jobs=24, keep_gcc_include_fixed=False, keep_gcc_intrin=False, logfile='_build/compile_commands.json', makefile=False, output_format='plist', output_path='/home/gregorj/Git/Experimental/codechecker-nasm/_reports', verbose='debug')
[DEBUG][2022-05-07 16:18:26] {system} [1630620] <140290582947648> - analyze.py:972 main() - Output will be stored to: '/home/gregorj/Git/Experimental/codechecker-nasm/_reports'
[DEBUG][2022-05-07 16:18:26] {buildlogger} [1630620] <140290582947648> - log_parser.py:420 get_compiler_includes() - Retrieving default includes via /usr/bin/c++ -E -x c++ - -v
[DEBUG][2022-05-07 16:18:26] {buildlogger} [1630620] <140290582947648> - log_parser.py:420 get_compiler_includes() - Retrieving default includes via /usr/bin/nasm -E -x '' - -v
[ERROR][2022-05-07 16:18:26] {buildlogger} [1630620] <140290582947648> - log_parser.py:1349 parse_unique_log() - The compile database is not valid.
[DEBUG][2022-05-07 16:18:26] {buildlogger} [1630620] <140290582947648> - log_parser.py:1350 parse_unique_log() - Traceback (most recent call last):
  File "/snap/codechecker/9/CodeChecker/lib/python3/codechecker_analyzer/buildlog/log_parser.py", line 1293, in parse_unique_log
    analyzer_clang_version)
  File "/snap/codechecker/9/CodeChecker/lib/python3/codechecker_analyzer/buildlog/log_parser.py", line 1076, in parse_options
    ImplicitCompilerInfo.set(details, compiler_info_file)
  File "/snap/codechecker/9/CodeChecker/lib/python3/codechecker_analyzer/buildlog/log_parser.py", line 574, in set
    iisk.compiler, iisk.language, iisk.compiler_flags),
  File "/snap/codechecker/9/CodeChecker/lib/python3/codechecker_analyzer/buildlog/log_parser.py", line 422, in get_compiler_includes
    include_dirs = ICI.__parse_compiler_includes(cmd)
  File "/snap/codechecker/9/CodeChecker/lib/python3/codechecker_analyzer/buildlog/log_parser.py", line 378, in __parse_compiler_includes
    lines = ImplicitCompilerInfo.__get_compiler_err(compile_cmd)
  File "/snap/codechecker/9/CodeChecker/lib/python3/codechecker_analyzer/buildlog/log_parser.py", line 349, in __get_compiler_err
    errors="ignore")
  File "/snap/codechecker/9/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/snap/codechecker/9/usr/lib/python3.6/subprocess.py", line 1295, in _execute_child
    restore_signals, start_new_session, preexec_fn)
TypeError: expected str, bytes or os.PathLike object, not NoneType

[DEBUG][2022-05-07 16:18:26] {buildlogger} [1630620] <140290582947648> - log_parser.py:1351 parse_unique_log() - expected str, bytes or os.PathLike object, not NoneType

Once I remove all nasm entries from the compilation database, everything works as expected.

Expected behaviour It would be nice if codechecker could ignore all nasm and yasm assembly tools.

Desktop (please complete the following information)

  • OS: Ubuntu 22.04

Thanks, Gregor

gjasny avatar May 07 '22 14:05 gjasny

Seems to be an serious bug, and diagnostics is a bit lacking (had to manually find the offending entry). Another example using clang:

[
{
  "directory": "/build/libs/libc/src",
  "command": "/usr/bin/clang  -nostdinc -o CMakeFiles/libc.dir/x86_64/memcpy.s.o -c /src/libs/libc/src/x86_64/memcpy.s",
  "file": "/src/libs/libc/src/x86_64/memcpy.s"
}
]

nolange avatar Jul 13 '22 07:07 nolange

I worked around by adding

-*.asm

to the skipfile.

gjasny avatar Jul 13 '22 15:07 gjasny

Well, I did run into it again - when trying to use the vscode codechecker plugin. I added a small bit to dump the arguments passed to POpen, the issue comes from ['/opt/cross-gcc/bin/x86_64-linux-gnu-gcc', '--sysroot=/opt/cross-gcc/x86_64-linux-gnu/sysroot', '-nostdinc', '-E', '-x', None, '-', '-v']

~/.local/bin/CodeChecker analyze ~/git/project/build/compile_commands.json -o /tmp/ --verbose debug

[DEBUG][2023-05-15 15:04:40] {system} [130460] <140701833670720> - analyze.py:990 main() - args: Namespace(input='/home/xxx/git/project/build/compile_commands.json', jobs=4, keep_gcc_include_fixed=False, keep_gcc_intrin=False, output_format='plist', makefile=False, compile_uniqueing='none', config_file=None, enable_z3='off', enable_z3_refutation='on', func=<function main at 0x7ff7b1b02160>, func_process_config_file=<function process_config_file at 0x7ff7b1b01c60>, output_path='/tmp', verbose='debug')
[DEBUG][2023-05-15 15:04:40] {system} [130460] <140701833670720> - analyze.py:991 main() - Output will be stored to: '/tmp'
[DEBUG][2023-05-15 15:04:40] {buildlogger} [130460] <140701833670720> - log_parser.py:420 get_compiler_includes() - Retrieving default includes via /opt/cross-gcc/bin/x86_64-linux-gnu-gcc --sysroot=/opt/cross-gcc/x86_64-linux-gnu/sysroot -nostdinc -E -x c - -v
[ERROR][2023-05-15 15:04:40] {buildlogger} [130460] <140701833670720> - log_parser.py:343 __get_compiler_err() - ['/opt/cross-gcc/bin/x86_64-linux-gnu-gcc', '--sysroot=/opt/cross-gcc/x86_64-linux-gnu/sysroot', '-nostdinc', '-E', '-x', 'c', '-', '-v']
[ERROR][2023-05-15 15:04:40] {buildlogger} [130460] <140701833670720> - log_parser.py:343 __get_compiler_err() - ['/opt/cross-gcc/bin/x86_64-linux-gnu-gcc', '/tmp/tmpb3lqbc2b.c']
[ERROR][2023-05-15 15:04:40] {buildlogger} [130460] <140701833670720> - log_parser.py:343 __get_compiler_err() - ['/opt/cross-gcc/bin/x86_64-linux-gnu-gcc', '-v']
[DEBUG][2023-05-15 15:04:40] {buildlogger} [130460] <140701833670720> - log_parser.py:420 get_compiler_includes() - Retrieving default includes via /opt/cross-gcc/bin/x86_64-linux-gnu-gcc --sysroot=/opt/cross-gcc/x86_64-linux-gnu/sysroot -nostdinc -E -x '' - -v
[ERROR][2023-05-15 15:04:40] {buildlogger} [130460] <140701833670720> - log_parser.py:343 __get_compiler_err() - ['/opt/cross-gcc/bin/x86_64-linux-gnu-gcc', '--sysroot=/opt/cross-gcc/x86_64-linux-gnu/sysroot', '-nostdinc', '-E', '-x', None, '-', '-v']
[ERROR][2023-05-15 15:04:40] {buildlogger} [130460] <140701833670720> - log_parser.py:1354 parse_unique_log() - The compile database is not valid.
[DEBUG][2023-05-15 15:04:40] {buildlogger} [130460] <140701833670720> - log_parser.py:1355 parse_unique_log() - Traceback (most recent call last):
  File "/home/xxx/.local/pipx/venvs/codechecker/lib/python3.11/site-packages/codechecker_analyzer/buildlog/log_parser.py", line 1289, in parse_unique_log
    action = parse_options(entry,
             ^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/.local/pipx/venvs/codechecker/lib/python3.11/site-packages/codechecker_analyzer/buildlog/log_parser.py", line 1076, in parse_options
    ImplicitCompilerInfo.set(details, compiler_info_file)
  File "/home/xxx/.local/pipx/venvs/codechecker/lib/python3.11/site-packages/codechecker_analyzer/buildlog/log_parser.py", line 575, in set
    'compiler_includes': ICI.get_compiler_includes(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/.local/pipx/venvs/codechecker/lib/python3.11/site-packages/codechecker_analyzer/buildlog/log_parser.py", line 424, in get_compiler_includes
    include_dirs = ICI.__parse_compiler_includes(cmd)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/.local/pipx/venvs/codechecker/lib/python3.11/site-packages/codechecker_analyzer/buildlog/log_parser.py", line 380, in __parse_compiler_includes
    lines = ImplicitCompilerInfo.__get_compiler_err(compile_cmd)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/.local/pipx/venvs/codechecker/lib/python3.11/site-packages/codechecker_analyzer/buildlog/log_parser.py", line 344, in __get_compiler_err
    proc = subprocess.Popen(
           ^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1024, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.11/subprocess.py", line 1834, in _execute_child
    self.pid = _fork_exec(
               ^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType

[DEBUG][2023-05-15 15:04:40] {buildlogger} [130460] <140701833670720> - log_parser.py:1356 parse_unique_log() - expected str, bytes or os.PathLike object, not NoneType

nolange avatar May 15 '23 13:05 nolange