Meshroom
Meshroom copied to clipboard
alicevision
Feature Matching with known camera positions fails without error report
We are currently developing a system to automatically measure the size and volume of packages. Our Setup consists of a few dozen cameras taking images every time a package is visible. Everything but the package itself is whited out to improve image processing. There already is a software in place detecting the exact position and rotation of the packages from a data matrix code on the packages. Now we tried to import the known camera positions into meshroom following this tutorial:
https://github.com/alicevision/meshroom/wiki/Using-known-camera-positions
Meshroom already works for us without setting the camera positions, but we hope to improve in processing time and precision by importing known data.
Feature Extraction and Image Matching work fine, but Feature Matching fails almost immediately (~ 0.05s) without any error message. All settings are default, except for those mentioned in the tutorial. Are we missing something here, or is there something we could change?
End of Feature Matching Log file:
[...]
[12:09:21.304391][info] loadPairs: image pair (343783022, 2081816213) added. File: "C:/Users/Cookie/Desktop/ILO Test/Test 7/MeshroomCache/ImageMatching/8a27a4d81e247841e984353105543b1cb05528c0/imageMatches.txt".
[12:09:21.304391][info] loadPairs: image pair (343783022, 2105747217) added. File: "C:/Users/Cookie/Desktop/ILO Test/Test 7/MeshroomCache/ImageMatching/8a27a4d81e247841e984353105543b1cb05528c0/imageMatches.txt".
[12:09:21.304391][info] loadPairs: image pair (343783022, 2112060982) added. File: "C:/Users/Cookie/Desktop/ILO Test/Test 7/MeshroomCache/ImageMatching/8a27a4d81e247841e984353105543b1cb05528c0/imageMatches.txt".
[12:09:21.304391][info] Number of pairs: 1790
[12:09:21.304391][info] There are 100 views and 1790 image pairs.
[12:09:21.304391][info] Load features and descriptors
Loading regions
0% 10 20 30 40 50 60 70 80 90 100%
|----|----|----|----|----|----|----|----|----|----|
[12:09:21.329397][info] Supported CUDA-Enabled GPU detected.
********************
Generated cameras.sfm
{
"version": [
"1",
"0",
"0"
],
"featuresFolders": [
"..\\..\\FeatureExtraction\\ecbf8625b58457c6fe2e7f24ddabe72a9f7f3f6f"
],
"matchesFolders": [
"..\\..\\FeatureMatching\\2270da2772965bb0a9ba7435196ba2043a8dd48f"
],
"views": [
{
"viewId": "52913916",
"poseId": "52913916",
"intrinsicId": "4228334583",
"resectionId": "3",
"path": "C:\/Users\/Cookie\/Desktop\/ILO Test\/Test 7\/Bilder\/screen_2560x2048_2021-07-12_11-21-03.jpg",
"width": "2560",
"height": "2048",
"metadata": {
"AliceVision:SensorWidthEstimation": "12.421158",
"AliceVision:useWhiteBalance": "1",
"Exif:ColorSpace": "65535",
"Exif:ExifVersion": "0232",
"Exif:FlashPixVersion": "0100",
"Exif:FocalLength": "12.5",
"Exif:FocalLengthIn35mmFilm": "34",
"Exif:YCbCrPositioning": "1",
"ResolutionUnit": "none",
"XResolution": "1",
"YResolution": "1",
"jpeg:subsampling": "4:2:0",
"oiio:ColorSpace": "sRGB"
}
},
[99 more views]
],
"intrinsics": [
{
"intrinsicId": "4228334583",
"width": "2560",
"height": "2048",
"sensorWidth": "12.421158228782492",
"sensorHeight": "9.9369265830259934",
"serialNumber": "C:\/Users\/Cookie\/Desktop\/ILO Test\/Test 7\/Bilder",
"type": "radial3",
"initializationMode": "estimated",
"pxInitialFocalLength": "2576.2492845352476",
"pxFocalLength": "3088.2386389176409",
"principalPoint": [
"1279.3075422814297",
"1023.2748619012708"
],
"distortionParams": [
"0.0071717341699994719",
"-0.0079749659609448329",
"0.019229333447597727"
],
"locked": "0"
}
],
"poses": [
{
"poseId": "52913916",
"pose": {
"transform": {
"rotation": [
"0.99982353353122455",
"0.018415941781845472",
"0.0037087579354952117",
"-0.018400868864194942",
"0.99982245430330619",
"-0.0040580655435683436",
"-0.0037828325602809396",
"0.0039891050626517334",
"0.99998488849513123"
],
"center": [
"0.49581668745113749",
"0.44840827961297747",
"-0.11616643035302546"
]
},
"locked": "0"
}
},
[99 more poses]
]
}
Edited cameras.sfm Note:
- The 0s in the view rotations are correct
- We already tried matching the original layout exactly (number of spaces and path formating), which didnt change anything
{
"version": [
"1",
"0",
"0"
],
"views": [
{
"viewId": "52913916",
"poseId": "52913916",
"intrinsicId": "4228334583",
"resectionId": "3",
"path": "C:/Users/Cookie/Desktop/ILO Test/Test 7/Bilder/screen_2560x2048_2021-07-12_11-21-03.jpg",
"width": "2560",
"height": "2048",
"metadata": {
"AliceVision:SensorWidthEstimation": "12.421158",
"AliceVision:useWhiteBalance": "1",
"Exif:ColorSpace": "65535",
"Exif:ExifVersion": "0232",
"Exif:FlashPixVersion": "0100",
"Exif:FocalLength": "12.5",
"Exif:FocalLengthIn35mmFilm": "34",
"Exif:YCbCrPositioning": "1",
"ResolutionUnit": "none",
"XResolution": "1",
"YResolution": "1",
"jpeg:subsampling": "4:2:0",
"oiio:ColorSpace": "sRGB"
}
},
[99 more views]
],
"intrinsics": [
{
"intrinsicId": "4228334583",
"width": "2560",
"height": "2048",
"sensorWidth": "12.421158228782492",
"sensorHeight": "9.9369265830259934",
"serialNumber": "C:/Users/Cookie/Desktop/ILO Test/Test 7/Bilder",
"type": "radial3",
"initializationMode": "estimated",
"pxInitialFocalLength": "2576.2492845352476",
"pxFocalLength": "3088.2386389176409",
"principalPoint": [
"1279.3075422814297",
"1023.2748619012708"
],
"distortionParams": [
"0.0071717341699994719",
"-0.0079749659609448329",
"0.019229333447597727"
],
"locked": "0"
}
],
"poses": [
{
"poseId": "52913916",
"pose": {
"transform": {
"rotation": [
"-0.2494173",
"0.9683961",
"0",
"-0.9683961",
"-0.2494173",
"0",
"0",
"0",
"1"
],
"center": [
"-0.984684",
"0.5461588",
"-5.3046665"
]
},
"locked": "1"
}
},
[99 more poses]
]
}
System
- OS: Win 10
- Python version 3.7.4
- Meshroom version: 2021.1.0 (Release version)
Hardware:
- CPU: AMD Ryzen 5 3600
- GPU: Nvidia GTX 1070
- RAM: 32 GB
Please share the log with the verbosity set to Trace so more informations can be shown and possibly help the debuggin
"featuresFolders": [ "..\..\FeatureExtraction\ecbf8625b58457c6fe2e7f24ddabe72a9f7f3f6f" ], "matchesFolders": [ "..\..\FeatureMatching\2270da2772965bb0a9ba7435196ba2043a8dd48f" ]
here is the log, thanks 0.log
I manage to reproduce the error on another dataset, it's a segfault, the StructureEstimationFromKnownPoses is probably broken. I'll try to have a deeper look in the next days
I manage to reproduce the error on another dataset, it's a segfault, the
StructureEstimationFromKnownPosesis probably broken. I'll try to have a deeper look in the next days
So, I think I found the problem. When I reproduce the same project on meshroom version 2020.1 I notice that when know poses position is selected, the ImagesMatching module makes less items compared to the default options, so the featureMatching node fails because it's expecting to have more items (it literally seeks for items that weren't create).
The reason? well it's because it's only processing the first 40 pictures and skipping the rest of it. Notice that feature extraction split the pictures in groups of 40.
I'm about to process everything by script and cmd now, I'll update as soon as I get the results.
ps: this problems seems to occur only with data set greater than 40 pictures.
well i actually reproduced the error on 2021.1 with only 3 images from the monstree mini dataset. On the other hand, launching the same command with the develop version it works fine. So either the bug has been resolved in the meantime or, as you say, it happens only for datasets larger than 40 images
well i actually reproduced the error on 2021.1 with only 3 images from the monstree mini dataset. On the other hand, launching the same command with the develop version it works fine. So either the bug has been resolved in the meantime or, as you say, it happens only for datasets larger than 40 images
Quick question. So, After i convert my euler rotation angle to quaternions shall I convert those 4 values into an orthogonal matrix? which values correspond to which one. there are 9 values being input into the rotation.
Yes you need to transform the quaternion into a rotation matrix. The rotation matrix for the pose is saved in colum-major order, so first 3 elements of the list are the first column, the second 3 elements the second column and so on.
Yes you need to transform the quaternion into a rotation matrix. The rotation matrix for the pose is saved in colum-major order, so first 3 elements of the list are the first column, the second 3 elements the second column and so on.
Thanks, that was helpful. Regarding the center position, does it refers to the center of the camera or the center of the picture(hence the deepth should be the location of the surface?)
The pose center is the position of the camera center, the point of projection. The picture center is on the image plane at distance f (the focal) from the camera center.
okay thanks. By the way, does the featuring matching node works even if you have enable the know position from camera poses option? it doesn't seems to be works for me unless i disable that option, but then I just get around 70% of the true size of the object.
I know the exact position of my cameras an rotation (Had an stepper motor moving the camera in a perfect circle-path around the object) so the x, and y coordinates are calculated by taking the sine and cosine of the angle relative to the origin of the circle times the radius of it. and the height is known because I used another step motor to move the camera up and down.
then the rotation is also known because the camera is always facing the object in the middle of the circle. probably i had some mistakes on the rotation angle but not matter what I tried the featuring matching node doesn't work if the option is enable.
Hello,
Regarding the issue with featureMatching crashing when using know poses: I have the same issue in Meshroom 2021.2.0, do you know if a fix is going to be released? or if there is an existing work-arround for working with know camera poses ?
Thanks a lot
Unfortunately there has been a big gap since the last release. I don't have a date but it's on its way.
I believe the rotation should be in row major order as per this script here https://github.com/evo-biomech/scAnt/blob/5a400521f78160f38d2558bd67eb58afa91f95c2/scripts/estimate_camera_positions.py#L293C2-L293C2
