TeleSculptor icon indicating copy to clipboard operation
TeleSculptor copied to clipboard

Crash with too few frames for depth map computation

Open predicative opened this issue 4 years ago • 1 comments

Running "Compute -> Batch Compute Depth Maps" when there are fewer frames than the configured support can result in a crash due to out-of-bounds data access. The suspect code is here:

https://github.com/Kitware/TeleSculptor/blob/e90a57a81db03c7e96ea5a0d51538974312ed2d2/gui/tools/ComputeAllDepthTool.cxx#L208-L213

If frames_in_range.size() is less than 2 * halfsupport, then num_frames will be negative, resulting in static_cast<size_t>(num_frames) being a large positive number and d->num_depth_maps inappropriately being the full value of d->num_depth. The main for loop then attempts out-of-bounds access to frames_in_range.

It would prevent the undefined behavior to just do something like if(num_frames < 0) num_frames = 0;, but perhaps the tool should log / report that it didn't do any work.

predicative avatar Jul 13 '20 18:07 predicative

Good catch. We should fix this on the release branch for the next patch release.

mleotta avatar Jul 13 '20 18:07 mleotta