sonar-cxx Cannot find a project file to assign the valgrind error when using meson

Description

I have a moderately complex meson setup that is reused by several projects and I find the valgrind (memcheck) results are not fed into sonarqube, giving the error "Cannot find a project file to assign the valgrind error".

I cannot share everything because some proprietary code is involved, but I'll try to sketch the situation as well as possible.

Repository layout

Main repository is in /home/username/repo/git/projectname
Sources are in subdirectories /home/username/repo/git/projectname/src
Includes are in subdirectories of /home/username/repo/git/projectname/include
Unit tests are in subdirectories of /home/username/repo/git/projectname/units
Builds are made in /home/username/repo/git/projectname/build

Sonar properties file

Located at /home/username/repo/git/projectname/build/meson_tools/sonarqube
Contents:

sonar.projectKey=projecname
sonar.projectName=projectname
sonar.projectVersion=3c9295e9+
sonar.sources=/home/username/repo/git/projectname/src,/home/username/repo/git/projectname/include,/home/username/repo/git/projectname/unit
sonar.cxx.valgrind.reportPath=/home/username/repo/git/projectname/build/meson_tools/sonarqube/memcheck.xml
sonar.cxx.includeDirectories=/home/username/repo/git/projectname/include,/home/username/repo/git/projecname/build,/usr/include/c++/8,/usr/include/x86_64-linux-gnu/c++/8,/usr/include/c++/8/backward,/usr/lib/gcc/x86_64-linux-gnu/8/include,/usr/lib/gcc/x86_64-linux-gnu/8/include-fixed,/usr/include/x86_64-linux-gnu,/usr/include
sonar.projectBaseDir=/home/username/repo/git/projectname
sonar.sourceEncoding=UTF-8

Valgrind

Valgrind output is generated in /home/username/repo/git/projectname/meson_tools/sonarqube/memcheck.xml
Paths in the xml are relative to the /home/username/repo/git/projectname

Sample valgrind error frame output

<error>
  <unique>0x0</unique>
  <tid>1</tid>
  <kind>Leak_DefinitelyLost</kind>
  <xwhat>
    <text>23 bytes in 1 blocks are definitely lost in loss record 1 of 9</text>
    <leakedbytes>23</leakedbytes>
    <leakedblocks>1</leakedblocks>
  </xwhat>
  <stack>
    <frame>
      <ip>0x483577F</ip>
      <obj>/usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>malloc</fn>
      <dir>./coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>299</line>
    </frame>
    <frame>
      <ip>0x5136FF9</ip>
      <obj>/lib/x86_64-linux-gnu/libc-2.28.so</obj>
      <fn>strdup</fn>
      <dir>/build/glibc-d2N3Ld/glibc-2.28/string</dir>
      <file>strdup.c</file>
      <line>42</line>
    </frame>
    <frame>
      <ip>0x57BBF0</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>PdfFormFixture::getTestPixelColour(PoDoFo::PdfMemDocument*)</fn>
      <dir>unit/utils</dir>
      <file>test_pdfForm.cpp</file>
      <line>177</line>
    </frame>
    <frame>
      <ip>0x572F46</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>(anonymous namespace)::____C_A_T_C_H____T_E_S_T____0::test()</fn>
      <dir>unit/utils</dir>
      <file>test_pdfForm.cpp</file>
      <line>366</line>
    </frame>
    <frame>
      <ip>0x577545</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>Catch::TestInvokerAsMethod&lt;(anonymous namespace)::____C_A_T_C_H____T_E_S_T____0&gt;::invoke() const</fn>
      <dir>unit/include/lib</dir>
      <file>catch.hpp</file>
      <line>823</line>
    </frame>
    <frame>
      <ip>0x326A82</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>Catch::TestCase::invoke() const</fn>
      <dir>unit/include/lib</dir>
      <file>catch.hpp</file>
      <line>11742</line>
    </frame>
    <frame>
      <ip>0x31EAB6</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>Catch::RunContext::invokeActiveTestCase()</fn>
      <dir>unit/include/lib</dir>
      <file>catch.hpp</file>
      <line>10601</line>
    </frame>
    <frame>
      <ip>0x31E66E</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>Catch::RunContext::runCurrentTest(std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;&amp;, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;&amp;)</fn>
      <dir>unit/include/lib</dir>
      <file>catch.hpp</file>
      <line>10574</line>
    </frame>
    <frame>
      <ip>0x31C8FB</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>Catch::RunContext::runTest(Catch::TestCase const&amp;)</fn>
      <dir>unit/include/lib</dir>
      <file>catch.hpp</file>
      <line>10344</line>
    </frame>
    <frame>
      <ip>0x32085D</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>Catch::(anonymous namespace)::runTests(std::shared_ptr&lt;Catch::Config&gt; const&amp;)</fn>
      <dir>unit/include/lib</dir>
      <file>catch.hpp</file>
      <line>10903</line>
    </frame>
    <frame>
      <ip>0x322707</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>Catch::Session::runInternal()</fn>
      <dir>unit/include/lib</dir>
      <file>catch.hpp</file>
      <line>11098</line>
    </frame>
    <frame>
      <ip>0x32239E</ip>
      <obj>/home/username/repo/git/projectname/build/units</obj>
      <fn>Catch::Session::run()</fn>
      <dir>unit/include/lib</dir>
      <file>catch.hpp</file>
      <line>11055</line>
    </frame>
  </stack>
</error>

** Sonar log snippet **

11:53:21.074 INFO: Sensor C++ (Community) ValgrindSensor [cxx]
11:53:21.074 INFO: Searching reports by relative path with basedir '/home/username/repo/git/projectname' and search prop 'sonar.cxx.valgrind.reportPath'
11:53:21.074 DEBUG: Normalized report includes to '[/home/username/repo/git/projectname/build/meson_tools/sonarqube/memcheck.xml]'
11:53:21.074 DEBUG: Scanner uses normalized report path(s): '/home/username/repo/git/projectname/build/meson_tools/sonarqube/memcheck.xml'
11:53:21.075 INFO: Parser will parse '1' report file(s)
11:53:21.075 INFO: Processing report '/home/username/repo/git/projectname/build/meson_tools/sonarqube/memcheck.xml'
11:53:21.075 DEBUG: Parsing 'Valgrind' format
11:53:21.082 WARN: Cannot find a project file to assign the valgrind error 'ValgrindError [kind=Leak_DefinitelyLost, text=23 bytes in 1 blocks are definitely lost in loss record 4 of 9, stacks=[ ValgrindStack=[0x483577F: malloc (vg_replace_malloc.c:299)
0x5136FF9: strdup (strdup.c:42)
0x57BBF0: PdfFormFixture::getTestPixelColour(PoDoFo::PdfMemDocument*) (test_pdfForm.cpp:177)
0x573AA0: (anonymous namespace)::____C_A_T_C_H____T_E_S_T____0::test() (test_pdfForm.cpp:427)
0x577545: Catch::TestInvokerAsMethod<(anonymous namespace)::____C_A_T_C_H____T_E_S_T____0>::invoke() const (catch.hpp:823)
0x326A82: Catch::TestCase::invoke() const (catch.hpp:11742)
0x31EAB6: Catch::RunContext::invokeActiveTestCase() (catch.hpp:10601)
0x31E66E: Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (catch.hpp:10574)
0x31C8FB: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:10344)
0x32085D: Catch::(anonymous namespace)::runTests(std::shared_ptr<Catch::Config> const&) (catch.hpp:10903)
0x322707: Catch::Session::runInternal() (catch.hpp:11098)
0x32239E: Catch::Session::run() (catch.hpp:11055)] ] ]' to
11:53:21.082 WARN: Cannot find a project file to assign the valgrind error 'ValgrindError [kind=Leak_DefinitelyLost, text=23 bytes in 1 blocks are definitely lost in loss record 1 of 9, stacks=[ ValgrindStack=[0x483577F: malloc (vg_replace_malloc.c:299)
0x5136FF9: strdup (strdup.c:42)
0x57BBF0: PdfFormFixture::getTestPixelColour(PoDoFo::PdfMemDocument*) (test_pdfForm.cpp:177)
0x572F46: (anonymous namespace)::____C_A_T_C_H____T_E_S_T____0::test() (test_pdfForm.cpp:366)
0x577545: Catch::TestInvokerAsMethod<(anonymous namespace)::____C_A_T_C_H____T_E_S_T____0>::invoke() const (catch.hpp:823)
0x326A82: Catch::TestCase::invoke() const (catch.hpp:11742)
0x31EAB6: Catch::RunContext::invokeActiveTestCase() (catch.hpp:10601)
0x31E66E: Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (catch.hpp:10574)
0x31C8FB: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:10344)
0x32085D: Catch::(anonymous namespace)::runTests(std::shared_ptr<Catch::Config> const&) (catch.hpp:10903)
0x322707: Catch::Session::runInternal() (catch.hpp:11098)
0x32239E: Catch::Session::run() (catch.hpp:11055)] ] ]' to
11:53:21.082 WARN: Cannot find a project file to assign the valgrind error 'ValgrindError [kind=Leak_DefinitelyLost, text=23 bytes in 1 blocks are definitely lost in loss record 2 of 9, stacks=[ ValgrindStack=[0x483577F: malloc (vg_replace_malloc.c:299)
0x5136FF9: strdup (strdup.c:42)
0x57BBF0: PdfFormFixture::getTestPixelColour(PoDoFo::PdfMemDocument*) (test_pdfForm.cpp:177)
0x573392: (anonymous namespace)::____C_A_T_C_H____T_E_S_T____0::test() (test_pdfForm.cpp:383)
0x577545: Catch::TestInvokerAsMethod<(anonymous namespace)::____C_A_T_C_H____T_E_S_T____0>::invoke() const (catch.hpp:823)
0x326A82: Catch::TestCase::invoke() const (catch.hpp:11742)
0x31EAB6: Catch::RunContext::invokeActiveTestCase() (catch.hpp:10601)
0x31E66E: Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (catch.hpp:10574)
0x31C8FB: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:10344)
0x32085D: Catch::(anonymous namespace)::runTests(std::shared_ptr<Catch::Config> const&) (catch.hpp:10903)
0x322707: Catch::Session::runInternal() (catch.hpp:11098)
0x32239E: Catch::Session::run() (catch.hpp:11055)] ] ]' to
11:53:21.083 WARN: Cannot find a project file to assign the valgrind error 'ValgrindError [kind=Leak_DefinitelyLost, text=23 bytes in 1 blocks are definitely lost in loss record 3 of 9, stacks=[ ValgrindStack=[0x483577F: malloc (vg_replace_malloc.c:299)
0x5136FF9: strdup (strdup.c:42)
0x57BBF0: PdfFormFixture::getTestPixelColour(PoDoFo::PdfMemDocument*) (test_pdfForm.cpp:177)
0x5736D3: (anonymous namespace)::____C_A_T_C_H____T_E_S_T____0::test() (test_pdfForm.cpp:399)
0x577545: Catch::TestInvokerAsMethod<(anonymous namespace)::____C_A_T_C_H____T_E_S_T____0>::invoke() const (catch.hpp:823)
0x326A82: Catch::TestCase::invoke() const (catch.hpp:11742)
0x31EAB6: Catch::RunContext::invokeActiveTestCase() (catch.hpp:10601)
0x31E66E: Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (catch.hpp:10574)
0x31C8FB: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:10344)
0x32085D: Catch::(anonymous namespace)::runTests(std::shared_ptr<Catch::Config> const&) (catch.hpp:10903)
0x322707: Catch::Session::runInternal() (catch.hpp:11098)
0x32239E: Catch::Session::run() (catch.hpp:11055)] ] ]' to
11:53:21.083 WARN: Cannot find a project file to assign the valgrind error 'ValgrindError [kind=Leak_DefinitelyLost, text=23 bytes in 1 blocks are definitely lost in loss record 5 of 9, stacks=[ ValgrindStack=[0x483577F: malloc (vg_replace_malloc.c:299)
0x5136FF9: strdup (strdup.c:42)
0x57BBF0: PdfFormFixture::getTestPixelColour(PoDoFo::PdfMemDocument*) (test_pdfForm.cpp:177)
0x573D66: (anonymous namespace)::____C_A_T_C_H____T_E_S_T____0::test() (test_pdfForm.cpp:441)
0x577545: Catch::TestInvokerAsMethod<(anonymous namespace)::____C_A_T_C_H____T_E_S_T____0>::invoke() const (catch.hpp:823)
0x326A82: Catch::TestCase::invoke() const (catch.hpp:11742)
0x31EAB6: Catch::RunContext::invokeActiveTestCase() (catch.hpp:10601)
0x31E66E: Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) (catch.hpp:10574)
0x31C8FB: Catch::RunContext::runTest(Catch::TestCase const&) (catch.hpp:10344)
0x32085D: Catch::(anonymous namespace)::runTests(std::shared_ptr<Catch::Config> const&) (catch.hpp:10903)
0x322707: Catch::Session::runInternal() (catch.hpp:11098)
0x32239E: Catch::Session::run() (catch.hpp:11055)] ] ]' to
11:53:21.083 DEBUG: CXX-VALGRIND processed = 0
11:53:21.083 INFO: CXX-VALGRIND processed = 0

May 03 '19 10:05 bartlibert

Hi @bartlibert,

I don't use the sensor so I can only guess:

The source code is here: https://github.com/SonarOpenCommunity/sonar-cxx/blob/master/cxx-sensors/src/main/java/org/sonar/cxx/sensors/valgrind/CxxValgrindSensor.java#L77
As you can see you get the message when the last frame from the stack is not in own code. Own means it must be a file which is below root and indexed.

In your sample it seams that files are from a library or in test code?

Regards,

May 04 '19 16:05 guwirth

Hi @guwirth, I already had a look at the code and I assumed that the "getLastOwnFrame" call would try to find a file within the source folder(s), hence it should find test_pdfForm.cpp and try to assign to that. Am I interpreting this wrong and is "getLastOwnFrame" badly named?

May 06 '19 06:05 bartlibert

Hi @bartlibert,

is test_pdfForm.cpp in/below source or test?

I'm not sure but test files are not handled as source files?
Please verify in the LOG file that file test_pdfForm.cpp is indexed.

Regards,

May 06 '19 07:05 guwirth

It is inside the unit folder (so the tests), which is also in the "sonar.sources" config variable, so I don't know if it should be handled differently? The file is indexed, although it has some syntax errors, related to the catch2 testing framework.

May 06 '19 09:05 bartlibert

tl;dr: Valgrind output MUST contain absolute paths for <dir> entries and they MUST start with whatever you put as projectBaseDir.

@guwirth Been debugging this for a few days... had basically the same issue as @bartlibert and I think I finally found the issue.

Inside getLastOwnFrame we loop over every frame in the valgrind stack trying to find one that's "owned" by our project (Bart was right about that part):https://github.com/SonarOpenCommunity/sonar-cxx/blob/4f9280bd1d0a114606914733dd71e0c1b5a6d87d/cxx-sensors/src/main/java/org/sonar/cxx/sensors/valgrind/ValgrindStack.java#L91-L98

You might get this far and think the issue of "ownership" has to do with whether or not the file is indexed or being excluded for some reason but you'd be wrong in chasing that lead. Instead it's this nasty snippet:

https://github.com/SonarOpenCommunity/sonar-cxx/blob/4f9280bd1d0a114606914733dd71e0c1b5a6d87d/cxx-sensors/src/main/java/org/sonar/cxx/sensors/valgrind/ValgrindStack.java#L38-L40

path.startsWith(folder) is checking against baseDir, which in my case was the source of the issue because the valgrind report that I generated was actually generated from a binary that was created in another directory and this sensor (woefully) has no support for relative paths in valgrind output.

May 20 '21 21:05 shua27

Hi @shua27,

thanks for your feedback.

First of all I like to ask you to read:

https://github.com/SonarOpenCommunity/sonar-cxx/wiki/Scan-Report-Files
https://github.com/SonarOpenCommunity/sonar-cxx/wiki/Troubleshooting-Reports
https://github.com/SonarOpenCommunity/sonar-cxx/wiki/sonar.cxx.valgrind.reportPaths

Does this make sense for Valgrind?

path.startsWith(folder) is checking against baseDir, which in my case was the source of the issue because the valgrind report that I generated was actually generated from a binary that was created in another directory and this sensor (woefully) has no support for relative paths in valgrind output.

If you create the reports on another computer, you need to adjust the paths in the report files to match the paths on the computer running the scanner.

Or is the problem the “has no support for relative paths in valgrind output“?

Do you have a sample report file / snippet showing the issue? Or even better: a patch for the issue?

I’m not familiar with Valgrind and don’t have the possibility to make own tests… but we are for sure interested to improve the sensor.

Regards,

May 22 '21 05:05 guwirth

To be fixed: … sensor (woefully) has no support for relative paths in valgrind output.

Apr 30 '22 05:04 guwirth

sonar-cxx sonar-cxx copied to clipboard

Cannot find a project file to assign the valgrind error when using meson

Description

sonar-cxx
sonar-cxx copied to clipboard