Fast-DDS-statistics-backend icon indicating copy to clipboard operation
Fast-DDS-statistics-backend copied to clipboard

Memory usage performance test [15785]

Open jparisu opened this issue 3 years ago • 8 comments

Merge after:

  • #175
  • #178
  • #172
  • #174
  • #177
  • #176

jparisu avatar Sep 30 '22 11:09 jparisu

Codecov Report

Base: 58.42% // Head: 58.42% // No change to project coverage :thumbsup:

Coverage data is based on head (d7bd7bf) compared to base (74f5197). Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #170   +/-   ##
=======================================
  Coverage   58.42%   58.42%           
=======================================
  Files          33       33           
  Lines        4421     4421           
  Branches     2352     2352           
=======================================
  Hits         2583     2583           
  Misses         54       54           
  Partials     1784     1784           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

codecov[bot] avatar Oct 25 '22 07:10 codecov[bot]

@EduPonz I think I don't have enough knowledge about bash scripting to review this one ...

MiguelCompany avatar Oct 26 '22 08:10 MiguelCompany

@jparisu Please rebase this one

MiguelCompany avatar Oct 27 '22 05:10 MiguelCompany

I have rebased the PR while looking at it. I think we ought to have a discussion about this; right now, it's a draft as it is far from being ready to be incorporated in CI and merged. The pain points I see with the current for merging as a test are:

  • [ ] The test creates a dependency on a specific Fast DDS example, which may not be present
  • [ ] CMakeLists.txt to declare the test for ctest
  • [ ] Discussion on passing/failing criteria
  • [ ] Script documentation & help guide
  • [ ] CSV documentation

Another possibility would be to add the script as a resource from which users can benefit, but I'm really attracted to the idea of having this kind of stress tests.

EduPonz avatar Oct 27 '22 06:10 EduPonz

Another possibility would be to add the script as a resource from which users can benefit, but I'm really attracted to the idea of having this kind of stress tests.

This test was never meant to be added in the CI. This was intended to be a manual test to have a partial idea of what is the memory usage of an execution. In order to automatize it and have a failing criteria I think it could not be done as results highly depend on the architecture, OS and state of the machine where tests are running. I think it would be better to have it as a helper script rather than an actual test (does Fast DDS have memory usage automatized tests with failing criteria?).

PD: I personally think we have bigger problems that having a stress test.

jparisu avatar Oct 27 '22 06:10 jparisu

This is an image of the results obtained by running this test: image

jparisu avatar Oct 27 '22 06:10 jparisu

I completely agree. I'd leave this as a draft for now and we'll see in the future

EduPonz avatar Oct 28 '22 07:10 EduPonz

@jparisu which is the status of this Draft? Does it make sense to try to merge it? I think we should change the status and assign a milestone so the discussion is had at some point.

JLBuenoLopez avatar Mar 16 '23 11:03 JLBuenoLopez