tesseract
tesseract copied to clipboard
tesseract-robotics
Trajectory Collision Logging
The goal of this is to improve the terminal output when performing a collision check on a trajectory. In doing this I also created objects for holding collision information from a trajectory that may be useful elsewhere.
Two examples of partial trajectory outputs from running the unit tests:
As compared to the previous output that was much harder to read:
The trajectory collision object provides a function that outputs a std::stringstream in the form of the table seen above. All the columns are adjusted to always be a constant width depending on the longest string in that column, so the output should always look clean, similar to what is done with the trajopt iterations.
Codecov Report
Merging #765 (1cdd248) into master (697c61a) will decrease coverage by
5.08%. The diff coverage is30.87%.
:exclamation: Current head 1cdd248 differs from pull request most recent head bdbc5d3. Consider uploading reports for the commit bdbc5d3 to get more accurate results
@@ Coverage Diff @@
## master #765 +/- ##
==========================================
- Coverage 94.01% 88.93% -5.08%
==========================================
Files 238 224 -14
Lines 14377 14041 -336
==========================================
- Hits 13516 12487 -1029
- Misses 861 1554 +693
| Impacted Files | Coverage Δ | |
|---|---|---|
| ...sion/core/include/tesseract_collision/core/types.h | 66.66% <0.00%> (-33.34%) |
:arrow_down: |
| tesseract_collision/core/src/types.cpp | 17.95% <0.00%> (-63.34%) |
:arrow_down: |
| tesseract_environment/src/utils.cpp | 97.88% <100.00%> (+0.98%) |
:arrow_up: |
... and 154 files with indirect coverage changes
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
Overall I like the PR. I have a concern about performance. Currently this functions is used a lot withing motion planning which can be called thousands of times within a single planning request. This was one of the reasons we only enabled the print out if the logLevel was set to debug. It would be good to retains the same functionality here were only the detailed results are calculated if the logLevel is set to the debug level.
@marrts Actually now that I have looked at it again. It looks like it is just storing already calculated data for reporting when the table is generated. The additional calculations or searching is only done during the table generation; is that correct?
Also think it would be good to add google benchmark to show the performance between the two is unchanged. I recently did this for the type erasure due to the same concern found here.
@marrts Actually now that I have looked at it again. It looks like it is just storing already calculated data for reporting when the table is generated. The additional calculations or searching is only done during the table generation; is that correct?
Yes, that Is correct. It is just storing data in vectors and only writes out data if the logging level is appropriate. It might actually be faster for whenever it's actually printing numerous collisions because it's all done in one print rather than doing a new print out every time a collision is found.
@marrts Just as a precaution do you mind creating a simple google benchmark for this?
@marrts Just as a precaution do you mind creating a simple google benchmark for this?
Yeah, I'll do that sometime soon. Totally understand why you'd want that.
Setting this to WIP while #887 is being reviewed. In my brief benchmark testing offline I think I might need to make some minor adjustments, but there are still some specifics I need to test to see what causes slow downs/speed ups
The other PR has been merged. Can you post the benchmark results pre and post for this PR?
Here is a breakdown of the comparison
Head to head comparison
Benchmark Name Before (ms) After (ms) Difference (ms) Percent Increase (%)
--------------------------------------------------------------- ------------- ------------ ----------------- ----------------------
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1/real_time 958.525 1132.474 173.949 18.148
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1-LVS/real_time 53415.452 61365.821 7950.369 14.884
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1/real_time 2141.454 2331.869 190.415 8.892
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1-LVS/real_time 78237.537 85749.644 7512.107 9.602
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1/real_time 1233.610 1303.061 69.451 5.630
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1-LVS/real_time 31865.112 30425.657 -1439.455 -4.517
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1/real_time 2172.345 2166.079 -6.266 -0.288
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1-LVS/real_time 36319.682 36273.560 -46.122 -0.127
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2/real_time 1774.321 1893.335 119.014 6.708
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2-LVS/real_time 109365.787 113239.523 3873.735 3.542
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2/real_time 2395.357 2414.410 19.053 0.795
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2-LVS/real_time 69267.677 69063.502 -204.175 -0.295
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2/real_time 2050.288 2000.837 -49.451 -2.412
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2-LVS/real_time 42432.114 41830.464 -601.649 -1.418
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2/real_time 2222.277 2267.603 45.325 2.040
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2-LVS/real_time 24746.297 25054.259 307.963 1.244
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3/real_time 2697.419 2768.631 71.212 2.640
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3-LVS/real_time 152650.427 159605.757 6955.330 4.556
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3/real_time 3098.050 3214.223 116.174 3.750
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3-LVS/real_time 89702.604 89714.976 12.372 0.014
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3/real_time 2556.322 2509.686 -46.636 -1.824
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3-LVS/real_time 53177.748 51872.366 -1305.382 -2.455
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3/real_time 2622.164 2691.075 68.911 2.628
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3-LVS/real_time 29518.500 28813.092 -705.408 -2.390
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4/real_time 2381.191 2402.004 20.812 0.874
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4-LVS/real_time 193320.079 206127.528 12807.450 6.625
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4/real_time 2569.024 2722.625 153.601 5.979
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4-LVS/real_time 107468.985 112412.079 4943.094 4.600
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4/real_time 2579.073 2628.740 49.667 1.926
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4-LVS/real_time 60139.593 60028.616 -110.977 -0.185
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4/real_time 2751.155 2816.641 65.486 2.380
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4-LVS/real_time 34124.432 33891.213 -233.220 -0.683
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5/real_time 2811.866 2979.250 167.384 5.953
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5-LVS/real_time 233499.887 246387.016 12887.129 5.519
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5/real_time 3037.149 3231.836 194.688 6.410
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5-LVS/real_time 135242.791 132030.651 -3212.140 -2.375
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5/real_time 3008.213 3043.110 34.897 1.160
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5-LVS/real_time 68719.396 68039.634 -679.762 -0.989
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5/real_time 3068.121 3172.493 104.372 3.402
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5-LVS/real_time 37203.360 38446.067 1242.707 3.340
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1-Debug/real_time 7844.538 7733.510 -111.027 -1.415
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1-LVS-Debug/real_time 258576.236 270928.048 12351.812 4.777
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1-Debug/real_time 7900.439 8407.108 506.669 6.413
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1-LVS-Debug/real_time 259967.680 279304.592 19336.912 7.438
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1-Debug/real_time 4651.440 4667.878 16.438 0.353
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1-LVS-Debug/real_time 78020.257 74852.111 -3168.146 -4.061
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1-Debug/real_time 4802.480 4880.675 78.195 1.628
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1-LVS-Debug/real_time 76405.189 77551.009 1145.820 1.500
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2-Debug/real_time 4717.377 5188.839 471.462 9.994
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2-LVS-Debug/real_time 274566.706 280828.635 6261.929 2.281
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2-Debug/real_time 5145.284 5360.875 215.591 4.190
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2-LVS-Debug/real_time 141643.317 145551.179 3907.862 2.759
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2-Debug/real_time 3875.399 4121.579 246.180 6.352
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2-LVS-Debug/real_time 78761.821 80816.706 2054.885 2.609
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2-Debug/real_time 3920.468 4230.157 309.689 7.899
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2-LVS-Debug/real_time 41985.231 43063.930 1078.699 2.569
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3-Debug/real_time 5237.172 5499.742 262.570 5.014
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3-LVS-Debug/real_time 301075.638 293744.975 -7330.663 -2.435
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3-Debug/real_time 5818.764 5665.131 -153.632 -2.640
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3-LVS-Debug/real_time 161477.800 153152.782 -8325.018 -5.156
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3-Debug/real_time 4480.699 4421.241 -59.458 -1.327
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3-LVS-Debug/real_time 99306.365 87119.298 -12187.067 -12.272
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3-Debug/real_time 4481.950 4580.369 98.419 2.196
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3-LVS-Debug/real_time 47367.423 49067.364 1699.941 3.589
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4-Debug/real_time 4635.015 3968.432 -666.583 -14.381
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4-LVS-Debug/real_time 330405.296 321441.957 -8963.339 -2.713
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4-Debug/real_time 4045.773 4136.255 90.481 2.236
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4-LVS-Debug/real_time 165809.418 169407.807 3598.389 2.170
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4-Debug/real_time 4061.806 4200.697 138.891 3.419
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4-LVS-Debug/real_time 94852.683 93013.583 -1839.101 -1.939
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4-Debug/real_time 3941.139 4180.947 239.808 6.085
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4-LVS-Debug/real_time 54092.447 50096.033 -3996.414 -7.388
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5-Debug/real_time 4284.311 4310.733 26.422 0.617
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5-LVS-Debug/real_time 352487.911 350560.782 -1927.130 -0.547
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5-Debug/real_time 4751.441 4502.429 -249.012 -5.241
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5-LVS-Debug/real_time 188754.613 183143.672 -5610.940 -2.973
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5-Debug/real_time 4102.390 4199.820 97.430 2.375
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5-LVS-Debug/real_time 99755.810 98739.024 -1016.786 -1.019
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5-Debug/real_time 5060.199 4275.059 -785.140 -15.516
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5-LVS-Debug/real_time 53017.718 52184.097 -833.621 -1.572
[['Debug', True]]
Average Increase (%) : 0.147
Max Increase (%) : 9.994
Max Decrease (%) : -15.516
[['Debug', False]]
Average Increase (%) : 2.832
Max Increase (%) : 18.148
Max Decrease (%) : -4.517
[['LVS', True]]
Average Increase (%) : 0.653
Max Increase (%) : 14.884
Max Decrease (%) : -12.272
[['LVS', False]]
Average Increase (%) : 2.326
Max Increase (%) : 18.148
Max Decrease (%) : -15.516
[['CONTINUOUS', True]]
Average Increase (%) : 2.930
Max Increase (%) : 18.148
Max Decrease (%) : -14.381
[['DISCRETE', True]]
Average Increase (%) : 0.049
Max Increase (%) : 7.899
Max Decrease (%) : -15.516
[['CONTINUOUS', True], ['Debug', True]]
Average Increase (%) : 0.519
Max Increase (%) : 9.994
Max Decrease (%) : -14.381
[['CONTINUOUS', True], ['Debug', False]]
Average Increase (%) : 5.341
Max Increase (%) : 18.148
Max Decrease (%) : -2.375
[['DISCRETE', True], ['Debug', True]]
Average Increase (%) : -0.226
Max Increase (%) : 7.899
Max Decrease (%) : -15.516
[['DISCRETE', True], ['Debug', False]]
Average Increase (%) : 0.323
Max Increase (%) : 5.630
Max Decrease (%) : -4.517
[['STATE_SOLVER', True]]
Average Increase (%) : 1.501
Max Increase (%) : 18.148
Max Decrease (%) : -14.381
[['JOINT_GROUP', True]]
Average Increase (%) : 1.478
Max Increase (%) : 9.602
Max Decrease (%) : -15.516
Total:
Average Increase (%) : 1.489
Max Increase (%) : 18.148
Max Decrease (%) : -15.516
Graph
@marrts is the -Debug with the print out and the those without you expect there to be no difference because all works should only happen debug is enabled?
@marrts is the -Debug with the print out and the those without you expect there to be no difference because all works should only happen debug is enabled?
I'm not entirely following what you're asking, but I expect there to be a difference between with and without debug enabled. With debug enabled it previously did 1 extra thing which was to print out every collision it found right after it found it. Now with debug enabled it does 3 extra things
- Fully allocates the memory to the trajectory logging structure I made
- Stores the contact map results into the trajectory logging structure
- Does all the calculations and printouts to the terminal
Overall, I think it should actually be much more efficient at writing to the terminal, particularly when there are very many collisions because it will only ever have to do 1 write to the terminal.
Sorry, I am trying to confirm that with this feature disabled that performance is not impacted. I am assuming that the items that don't include -Debug or -D in the name has this feature disabled. If that is the case then it looks like there is a performance hit even with it disabled.
IMO the drastic improvement in collision output readability is well worth the very marginal performance penalty
Sorry, I am trying to confirm that with this feature disabled that performance is not impacted. I am assuming that the items that don't include -Debug or -D in the name has this feature disabled. If that is the case then it looks like there is a performance hit even with it disabled.
All the summaries at the end break down how it performed given each scenario, so you can specifically see how it performed with debugging disabled here:
[['Debug', False]]
Average Increase (%) : 2.832
Max Increase (%) : 18.148
Max Decrease (%) : -4.517
Running it multiple times seems to give slightly different results and it's hard to control enough variables on my laptop in running it so that I can do a perfect head-to-head comparison.
Also, I think there are other opportunities for improvements. The difference between using a state solver versus a joint group is very surprising to me and is evident by every other bar grouping in the bar chart being higher on average.
Additionally, I am confused as to what exactly the need for the addInterpolatedResults is here. I know if I remove those and all the contacts.push_back()s I get a average of a 5% reduction in runtime and my data-structure retains all the information.
On top of that this could be parallelized.
Also for reference, here is me running the current origin/master back-to-back, so demonstrating no code changes to get a sense of the noise in the benchmarking.
Benchmark Name Before (ms) After (ms) Difference (ms) Percent Increase (%)
--------------------------------------------------------------- ------------- ------------ ----------------- ----------------------
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1/real_time 1003.657 1068.008 64.351 6.412
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1-LVS/real_time 56894.056 59296.799 2402.743 4.223
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1/real_time 2283.373 2323.250 39.877 1.746
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1-LVS/real_time 84581.729 87431.428 2849.699 3.369
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1/real_time 1277.422 1294.082 16.660 1.304
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1-LVS/real_time 31179.620 30160.313 -1019.307 -3.269
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1/real_time 2123.891 2212.011 88.120 4.149
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1-LVS/real_time 37415.647 36585.026 -830.621 -2.220
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2/real_time 1804.995 1789.368 -15.627 -0.866
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2-LVS/real_time 111020.432 110841.641 -178.791 -0.161
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2/real_time 2330.248 2304.537 -25.710 -1.103
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2-LVS/real_time 68725.827 67318.537 -1407.289 -2.048
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2/real_time 1911.110 1879.689 -31.421 -1.644
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2-LVS/real_time 42400.442 42236.639 -163.804 -0.386
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2/real_time 2109.533 2029.039 -80.494 -3.816
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2-LVS/real_time 24945.525 23986.936 -958.588 -3.843
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3/real_time 2849.526 2658.909 -190.617 -6.689
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3-LVS/real_time 159492.696 158227.487 -1265.210 -0.793
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3/real_time 3136.433 3129.567 -6.866 -0.219
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3-LVS/real_time 89078.350 89293.703 215.353 0.242
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3/real_time 2449.430 2386.093 -63.337 -2.586
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3-LVS/real_time 54297.646 51190.203 -3107.443 -5.723
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3/real_time 2572.338 2515.724 -56.614 -2.201
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3-LVS/real_time 29644.480 29597.100 -47.379 -0.160
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4/real_time 2321.533 2290.501 -31.032 -1.337
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4-LVS/real_time 202775.704 201142.736 -1632.968 -0.805
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4/real_time 2611.087 2583.157 -27.930 -1.070
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4-LVS/real_time 113345.177 111562.205 -1782.972 -1.573
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4/real_time 2563.692 2502.231 -61.461 -2.397
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4-LVS/real_time 62337.623 61739.584 -598.039 -0.959
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4/real_time 2704.181 2614.099 -90.082 -3.331
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4-LVS/real_time 35097.266 34524.796 -572.470 -1.631
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5/real_time 2969.809 3001.505 31.696 1.067
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5-LVS/real_time 333519.783 248390.878 -85128.905 -25.524
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5/real_time 3984.166 3090.895 -893.271 -22.421
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5-LVS/real_time 180029.631 131332.950 -48696.681 -27.049
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5/real_time 4517.917 2918.976 -1598.941 -35.391
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5-LVS/real_time 95412.238 73293.973 -22118.265 -23.182
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5/real_time 3394.703 2974.535 -420.168 -12.377
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5-LVS/real_time 38763.180 37759.795 -1003.385 -2.589
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1-Debug/real_time 8419.713 8138.198 -281.515 -3.344
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1-LVS-Debug/real_time 334683.152 272195.455 -62487.697 -18.671
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1-Debug/real_time 9124.459 8282.076 -842.383 -9.232
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1-LVS-Debug/real_time 303948.858 276058.120 -27890.738 -9.176
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1-Debug/real_time 5264.208 4680.801 -583.407 -11.083
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1-LVS-Debug/real_time 77839.976 77690.022 -149.954 -0.193
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1-Debug/real_time 4981.283 4829.839 -151.444 -3.040
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1-LVS-Debug/real_time 85567.311 80324.772 -5242.539 -6.127
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2-Debug/real_time 5310.473 5040.656 -269.817 -5.081
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2-LVS-Debug/real_time 274630.213 284141.015 9510.803 3.463
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2-Debug/real_time 5142.592 5209.719 67.127 1.305
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2-LVS-Debug/real_time 144520.300 145727.033 1206.733 0.835
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2-Debug/real_time 3815.561 3964.266 148.705 3.897
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2-LVS-Debug/real_time 81005.128 83316.973 2311.845 2.854
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2-Debug/real_time 4006.639 4113.929 107.290 2.678
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2-LVS-Debug/real_time 42170.717 42933.659 762.942 1.809
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3-Debug/real_time 4992.196 5310.536 318.340 6.377
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3-LVS-Debug/real_time 290070.484 299201.157 9130.674 3.148
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3-Debug/real_time 5367.510 5583.383 215.873 4.022
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3-LVS-Debug/real_time 166893.670 152281.682 -14611.988 -8.755
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3-Debug/real_time 4931.994 4331.965 -600.029 -12.166
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3-LVS-Debug/real_time 98970.379 89932.807 -9037.572 -9.132
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3-Debug/real_time 4710.823 4416.300 -294.523 -6.252
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3-LVS-Debug/real_time 46597.905 48130.487 1532.582 3.289
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4-Debug/real_time 4238.859 3802.632 -436.227 -10.291
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4-LVS-Debug/real_time 339850.943 324295.497 -15555.446 -4.577
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4-Debug/real_time 4422.551 4007.207 -415.344 -9.391
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4-LVS-Debug/real_time 165505.750 170847.340 5341.590 3.227
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4-Debug/real_time 3957.293 4160.720 203.428 5.141
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4-LVS-Debug/real_time 98399.211 96283.930 -2115.281 -2.150
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4-Debug/real_time 4703.953 4091.664 -612.289 -13.016
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4-LVS-Debug/real_time 54625.249 51386.795 -3238.454 -5.928
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5-Debug/real_time 4562.593 4209.010 -353.583 -7.750
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5-LVS-Debug/real_time 357748.878 352148.110 -5600.769 -1.566
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5-Debug/real_time 4555.721 4421.076 -134.644 -2.955
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5-LVS-Debug/real_time 186668.137 183910.644 -2757.493 -1.477
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5-Debug/real_time 4629.003 4287.399 -341.604 -7.380
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5-LVS-Debug/real_time 94793.887 102145.711 7351.824 7.756
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5-Debug/real_time 4167.170 4330.145 162.975 3.911
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5-LVS-Debug/real_time 52307.787 53424.562 1116.775 2.135
[['Debug', True]]
Average Increase (%) : -2.822
Max Increase (%) : 7.756
Max Decrease (%) : -18.671
[['Debug', False]]
Average Increase (%) : -4.421
Max Increase (%) : 6.412
Max Decrease (%) : -35.391
[['LVS', True]]
Average Increase (%) : -3.333
Max Increase (%) : 7.756
Max Decrease (%) : -27.049
[['LVS', False]]
Average Increase (%) : -3.910
Max Increase (%) : 6.412
Max Decrease (%) : -35.391
[['CONTINUOUS', True]]
Average Increase (%) : -3.612
Max Increase (%) : 6.412
Max Decrease (%) : -27.049
[['DISCRETE', True]]
Average Increase (%) : -3.631
Max Increase (%) : 7.756
Max Decrease (%) : -35.391
[['CONTINUOUS', True], ['Debug', True]]
Average Increase (%) : -3.494
Max Increase (%) : 6.377
Max Decrease (%) : -18.671
[['CONTINUOUS', True], ['Debug', False]]
Average Increase (%) : -3.730
Max Increase (%) : 6.412
Max Decrease (%) : -27.049
[['DISCRETE', True], ['Debug', True]]
Average Increase (%) : -2.150
Max Increase (%) : 7.756
Max Decrease (%) : -13.016
[['DISCRETE', True], ['Debug', False]]
Average Increase (%) : -5.113
Max Increase (%) : 4.149
Max Decrease (%) : -35.391
[['STATE_SOLVER', True]]
Average Increase (%) : -3.986
Max Increase (%) : 7.756
Max Decrease (%) : -35.391
[['JOINT_GROUP', True]]
Average Increase (%) : -3.257
Max Increase (%) : 4.149
Max Decrease (%) : -27.049
Total:
Average Increase (%) : -3.622
Max Increase (%) : 7.756
Max Decrease (%) : -35.391
Graph
IMO the drastic improvement in collision output readability is well worth the very marginal performance penalty
I don't think 14% or 18% increase in some case is marginal. I would want to understand why these few have such a large performance hit.
Additionally, I am confused as to what exactly the need for the
addInterpolatedResultsis here. I know if I remove those and all thecontacts.push_back()s I get a average of a 5% reduction in runtime and my data-structure retains all the information.
The comment above the line code explains why it is needed. When interpolating at a finer resolution the cc_type and cc_time need to be updated to correctly represent where it is between the trajectory states.
Related the noise you are seeing. When running the benchmark it prints out a message state what to do around setting the cpu to performance. I always do this when running benchmarks because it helps reduce the noise a lot.
I may have time this weekend to pull it down on my machine and run it.
Updated results from all your suggestions (I also added the before/after avg load from the CPU if that is helpful):
Benchmark Name Before (ms) After (ms) Difference (ms) Percent Increase (%) Before Load Avg After Load Avg
--------------------------------------------------------------- ------------- ------------ ----------------- ---------------------- ----------------- ----------------
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1/real_time 1066.239 1045.404 -20.835 -1.954 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1-LVS/real_time 57575.411 57119.334 -456.078 -0.792 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1/real_time 2267.099 2274.193 7.095 0.313 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1-LVS/real_time 85575.841 83395.067 -2180.773 -2.548 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1/real_time 1266.696 1259.519 -7.178 -0.567 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1-LVS/real_time 29491.542 30482.810 991.268 3.361 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1/real_time 2017.350 2006.413 -10.938 -0.542 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1-LVS/real_time 35817.782 35732.625 -85.157 -0.238 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2/real_time 1794.332 1866.773 72.441 4.037 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2-LVS/real_time 110369.019 117054.118 6685.099 6.057 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2/real_time 2300.324 2401.338 101.014 4.391 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2-LVS/real_time 67839.022 71145.742 3306.720 4.874 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2/real_time 1864.075 1926.136 62.061 3.329 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2-LVS/real_time 41594.421 41642.760 48.339 0.116 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2/real_time 2108.580 2081.909 -26.671 -1.265 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2-LVS/real_time 23681.009 24370.145 689.136 2.910 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3/real_time 2689.133 2824.303 135.169 5.026 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3-LVS/real_time 155735.826 159247.638 3511.812 2.255 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3/real_time 3044.747 3117.507 72.759 2.390 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3-LVS/real_time 87616.419 89633.779 2017.360 2.302 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3/real_time 2363.430 2366.764 3.334 0.141 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3-LVS/real_time 52917.222 52369.086 -548.135 -1.036 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3/real_time 2507.343 2489.557 -17.786 -0.709 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3-LVS/real_time 29636.823 29847.450 210.627 0.711 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4/real_time 2302.919 2298.723 -4.196 -0.182 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4-LVS/real_time 201646.097 205019.809 3373.711 1.673 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4/real_time 2611.066 2590.732 -20.334 -0.779 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4-LVS/real_time 110692.843 110382.293 -310.550 -0.281 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4/real_time 2480.430 2502.223 21.794 0.879 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4-LVS/real_time 61136.226 60405.850 -730.376 -1.195 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4/real_time 2609.838 2654.163 44.325 1.698 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4-LVS/real_time 34145.410 34344.055 198.645 0.582 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5/real_time 2879.073 2932.705 53.632 1.863 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5-LVS/real_time 240072.061 244586.412 4514.351 1.880 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5/real_time 3067.519 3495.313 427.794 13.946 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5-LVS/real_time 130157.224 139024.873 8867.649 6.813 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5/real_time 2864.244 2919.160 54.916 1.917 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5-LVS/real_time 70290.624 68835.925 -1454.699 -2.070 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5/real_time 3033.402 3015.752 -17.651 -0.582 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5-LVS/real_time 38075.987 37721.393 -354.594 -0.931 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1-Debug/real_time 7958.135 8042.360 84.225 1.058 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ1-LVS-Debug/real_time 264689.774 266425.004 1735.230 0.656 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1-Debug/real_time 8165.495 8202.026 36.531 0.447 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ1-LVS-Debug/real_time 267958.867 291642.188 23683.321 8.838 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1-Debug/real_time 4624.190 4684.601 60.410 1.306 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ1-LVS-Debug/real_time 75908.785 77165.561 1256.776 1.656 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1-Debug/real_time 4764.798 4813.210 48.412 1.016 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ1-LVS-Debug/real_time 79656.058 78776.925 -879.133 -1.104 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2-Debug/real_time 5112.252 5038.922 -73.330 -1.434 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ2-LVS-Debug/real_time 285587.045 277804.558 -7782.487 -2.725 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2-Debug/real_time 5243.431 5116.900 -126.531 -2.413 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ2-LVS-Debug/real_time 149776.375 143916.522 -5859.853 -3.912 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2-Debug/real_time 3936.712 3939.538 2.826 0.072 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ2-LVS-Debug/real_time 82639.806 81957.018 -682.789 -0.826 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2-Debug/real_time 4121.298 3970.080 -151.218 -3.669 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ2-LVS-Debug/real_time 43651.903 43332.208 -319.695 -0.732 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3-Debug/real_time 5394.427 5309.353 -85.073 -1.577 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ3-LVS-Debug/real_time 299081.160 293036.851 -6044.309 -2.021 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3-Debug/real_time 5620.483 5881.330 260.847 4.641 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ3-LVS-Debug/real_time 154325.670 151100.090 -3225.579 -2.090 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3-Debug/real_time 4394.316 4357.203 -37.113 -0.845 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ3-LVS-Debug/real_time 88910.292 88601.622 -308.670 -0.347 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3-Debug/real_time 4389.569 4331.405 -58.164 -1.325 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ3-LVS-Debug/real_time 47513.696 48473.273 959.577 2.020 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4-Debug/real_time 3781.373 3793.360 11.987 0.317 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ4-LVS-Debug/real_time 327447.811 311533.700 -15914.111 -4.860 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4-Debug/real_time 4287.974 3962.655 -325.318 -7.587 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ4-LVS-Debug/real_time 170034.897 166316.167 -3718.730 -2.187 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4-Debug/real_time 4133.011 4088.650 -44.361 -1.073 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ4-LVS-Debug/real_time 96977.999 93980.264 -2997.735 -3.091 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4-Debug/real_time 4320.390 4062.093 -258.297 -5.979 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ4-LVS-Debug/real_time 53253.507 51480.666 -1772.841 -3.329 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5-Debug/real_time 4218.218 4189.873 -28.345 -0.672 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_STATE_SOLVER-TRAJ5-LVS-Debug/real_time 353624.833 347867.797 -5757.036 -1.628 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5-Debug/real_time 4516.150 4358.914 -157.236 -3.482 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_CONTINUOUS_JOINT_GROUP-TRAJ5-LVS-Debug/real_time 186899.796 180854.198 -6045.598 -3.235 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5-Debug/real_time 4137.428 4245.782 108.354 2.619 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_STATE_SOLVER-TRAJ5-LVS-Debug/real_time 100100.509 104858.205 4757.695 4.753 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5-Debug/real_time 4420.524 4400.378 -20.146 -0.456 4.48, 2.45, 1.42 4.13, 3.83, 2.32
BM_CHECK_TRAJ_DISCRETE_JOINT_GROUP-TRAJ5-LVS-Debug/real_time 53737.132 53899.459 162.326 0.302 4.48, 2.45, 1.42 4.13, 3.83, 2.32
[['Debug', True]]
Average Increase (%) : -0.822
Max Increase (%) : 8.838
Max Decrease (%) : -7.587
[['Debug', False]]
Average Increase (%) : 1.445
Max Increase (%) : 13.946
Max Decrease (%) : -2.548
[['LVS', True]]
Average Increase (%) : 0.265
Max Increase (%) : 8.838
Max Decrease (%) : -4.860
[['LVS', False]]
Average Increase (%) : 0.358
Max Increase (%) : 13.946
Max Decrease (%) : -7.587
[['CONTINUOUS', True]]
Average Increase (%) : 0.686
Max Increase (%) : 13.946
Max Decrease (%) : -7.587
[['DISCRETE', True]]
Average Increase (%) : -0.063
Max Increase (%) : 4.753
Max Decrease (%) : -5.979
[['CONTINUOUS', True], ['Debug', True]]
Average Increase (%) : -1.193
Max Increase (%) : 8.838
Max Decrease (%) : -7.587
[['CONTINUOUS', True], ['Debug', False]]
Average Increase (%) : 2.564
Max Increase (%) : 13.946
Max Decrease (%) : -2.548
[['DISCRETE', True], ['Debug', True]]
Average Increase (%) : -0.452
Max Increase (%) : 4.753
Max Decrease (%) : -5.979
[['DISCRETE', True], ['Debug', False]]
Average Increase (%) : 0.326
Max Increase (%) : 3.361
Max Decrease (%) : -2.070
[['STATE_SOLVER', True]]
Average Increase (%) : 0.402
Max Increase (%) : 6.057
Max Decrease (%) : -4.860
[['JOINT_GROUP', True]]
Average Increase (%) : 0.221
Max Increase (%) : 13.946
Max Decrease (%) : -7.587
Total
Average Increase (%) : 0.311
Max Increase (%) : 13.946
Max Decrease (%) : -7.587
Graph
Some of these results are interesting. Is the state solver and joint group benchmarks running the same code where the only different is the joint group and state solver?
Some of these results are interesting. Is the state solver and joint group benchmarks running the same code where the only different is the joint group and state solver?
Yup, I was very surprised to see that result as well, you can see the benchmark here
Hey just wanted to check; did you build in release because some of the times look very high?
Hey just wanted to check; did you build in release because some of the times look very high?
I’m not sure. I’m about to go out of town so I can’t check until late next week. If I had to guess it was probably release with debug, but definitely test it on your machine too. I was getting around these times consistently.
If I had to guess it was probably release with debug
This is indeed how it is built.
Is there anything else you need from me before this is ready to be merged?
If I had to guess it was probably release with debug
This is indeed how it is built.
Is there anything else you need from me before this is ready to be merged?
Are the units correct? Some have an average time 100,000ms which is over a minute.
Are the units correct? Some have an average time 100,000ms which is over a minute.
Oops, no they are not. These should say microseconds, not milliseconds.