AliceVision
AliceVision copied to clipboard
Add software to filter SfM data and to create landmarks-based masking for depth map computation
Linked to https://github.com/alicevision/Meshroom/pull/2211
Description
This PR introduces the following two main features:
- Depth maps computation can be masked using landmarks from SfM data.
- SfM data can be filtered to efficiently be used for depth maps masking.
The SfM data filtering supports the following features (the parameters are independent):
- Filtering landmarks over 3 steps:
- Step 1: Remove landmarks that have an insufficient number of landmarks.
Parameters:
minNbObservationsPerLandmark - Step 2: Remove landmarks with dissimilar observations compared to their neighbors in 3D.
Parameters:
nbNeighbors3D,minNbObservationsPerLandmark - Step 3: Remove landmarks with worse resolution than their neighbors in 3D. The resolution is based on pixel size.
Parameters:
radiusScale,useFeatureScale
- Step 1: Remove landmarks that have an insufficient number of landmarks.
Parameters:
- Filtering observations to select best observations per landmark and to ensure consistency with neighbors in 3D.
The observations are scored in function of distance between corresponding camera centers and landmarks positions.
The scores are iteratively propagated between neighboring landmarks in 3D.
Parameters:
maxNbObservationsPerLandmark,propagationEnabled,nbNeighbors3D,nbIterations,dampingEnabled,dampingFactor - Filtering observations (and eventually landmarks) to ensure non-sparsity in 2D projection of landmarks separately in each view.
Mask radius for projected landmarks per view is estimated and reported in an output text file (
outputRadiiFile). Parameters:nbNeighbors2D,percentile,maskRadiusThreshold
Future Work
- [ ] fix duplicate landmarks' biased influence.
- [ ] take into consideration the angle of observation when scoring the observations in addition to the distance to camera.
- [ ] include blur estimation.
- [ ] have a more complex relative estimation for neighborhoods (e.g. elliptic neighborhood with orientation, based on landmarks segmentation/classification for surface information...).
- [ ] verify if modifications are required in case of multiple capturing devices.