opencv_contrib
opencv_contrib copied to clipboard
opencv
Sfm and Viz in sample scene reconstruction
Opencv 3.2.0-dev Windows 10 VS 2015 VTK 7.1.0
I cannot retrieve camera pose in sample scenereconstruction of module sfm.
In images given in folder data it is not easy to see problem. I use my own data in this folder
@PhilLab @edgarriba
If you run sample :
sfm-example-scene_reconstruction.exe imagessfm.txt 2763 517 501 for left camera I can see this in viz window for left camera:
I think it is wrong there is no image with this pose im my image list.
Now If I change this line in scenereconstruction
path.push_back(Affine3d(Rs_est[i],ts_est[i]));
in
path.push_back(Affine3d(Rs_est[i],ts_est[i]).inv());
Now questions Does problem in image given exist or Is it me ? If problem exist What is wrong matrix camera pose or pose must be inversed for viz?
Thanks for your answer.
The camera pose rendering always crashes on my system (Win x64 Visual Studio 2013) so I had to comment it out. I also had problems with the Sagrada Familia tutorial getting a completely wrong reconstruction (points collapsed to two clusters) using all four images and an okayish result when only using three (I cannot remember which three I used). I also had to rotate the images
With my own scenes I was never able to get a plausible reconstruction. Before #1169 I always had the points collapse to several clusters and after this PR, they are a little more scattered but still not meaningful.
Before your PR it was possible. But that's not the problem. You want this parameter true and me false. I think a new method is necessary.
If you want I can help you to solve your problem. Sfm in opencv is good tool (VS 2015 windows 10) and I haven't got problem using my intrinsic and distorsion parameters. I inserted a new example without matching have you try it?
Tried it but had to comment one of the cv::reconstruct because I built OpenCV before your PR #1159 . Result looks good for both reconstructions but camera rendering still crashes.
P.S.: For the first reconstruction I had 26 times E0620 16:52:36.205718 6124 levenberg_marquardt.h:138] Failed to solve ,
for the second one:
E0620 16:54:32.270113 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 0052.006 0126.561 -815.862
E0620 16:54:32.271966 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 020.2712 048.4514 -314.097
E0620 16:54:32.271966 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 0034.473 049.3099 -414.716
E0620 16:54:32.272970 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 19.2406 27.8848 -232.27
E0620 16:54:32.614789 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 0236.201 0396.251 -2998.34
E0620 16:54:32.615263 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 00435.84 0807.843 -5869.42
E0620 16:54:32.619555 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 0124.831 0204.694 -1150.81
E0620 16:54:32.620031 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 011.7237 028.6944 -184.512
E0620 16:54:32.620031 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 06.75778 016.2415 -105.033
E0620 16:54:32.621094 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 039.8539 045.7767 -315.889
E0620 16:54:32.621094 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 010.4262 0014.881 -125.305
E0620 16:54:32.621094 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 06.94179 010.0408 -83.7256
E0620 16:54:32.625104 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 01006.83 01236.15 -11467.2
E0620 16:54:32.626106 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 00157.19 0211.043 -1981.69
E0620 16:54:32.626106 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 093.9546 0141.894 -1317.92
E0620 16:54:32.626106 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 078.7485 0135.644 -1219.36
E0620 16:54:32.627110 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 088.1175 0171.789 -1469.85
E0620 16:54:32.627110 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 00163.43 0349.813 -2822.05
E0620 16:54:32.628111 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 0254.897 00313.92 -2912.37
E0620 16:54:32.628111 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 079.6845 00107.57 -1010.13
E0620 16:54:32.628111 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 49.6629 075.491 -701.33
E0620 16:54:32.629114 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 039.9551 069.2346 -622.914
E0620 16:54:32.629114 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 039.4738 0077.299 -662.567
E0620 16:54:32.629114 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 050.1897 0107.733 -871.421
E0620 16:54:32.630117 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 104.636 235.304 00-1792
E0620 16:54:32.630117 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 0322.173 0391.024 -3570.66
E0620 16:54:32.631119 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 97.9004 127.708 -1163.3
E0620 16:54:32.631119 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 063.2273 090.6285 -808.212
E0620 16:54:32.631119 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 052.9399 084.3416 -724.989
E0620 16:54:32.632122 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 054.1729 095.7214 -784.791
E0620 16:54:32.632122 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 071.4151 0137.718 -1071.76
E0620 16:54:32.632122 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 0170.249 0348.625 -2576.95
E0620 16:54:32.633132 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 0494.605 0612.118 -5323.41
E0620 16:54:32.634130 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 00289.84 0382.567 -3209.35
E0620 16:54:32.634130 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 0298.242 0425.495 -3417.27
E0620 16:54:32.634130 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 0618.447 0959.271 -7355.26
E0620 16:54:32.637135 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 046.0257 077.7661 -495.375
E0620 16:54:32.637135 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 011.1575 031.7033 -236.946
E0620 16:54:32.637135 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 07.34207 020.7808 -157.089
E0620 16:54:32.638139 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 034.0044 038.0277 -307.645
E0620 16:54:32.638139 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 108.799 118.524 -951.48
E0620 16:54:32.639142 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 011.9086 017.2272 -178.623
E0620 16:54:32.640177 12744 intersect.cc:159] POINT BEHIND CAMERA 2: 08.50309 012.6373 -130.322
E0620 16:54:32.649202 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 0052.006 0126.561 -815.862
E0620 16:54:32.649202 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 020.2712 048.4514 -314.097
E0620 16:54:32.650388 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 0034.473 049.3099 -414.716
E0620 16:54:32.651051 12744 intersect.cc:159] POINT BEHIND CAMERA 1: 19.2406 27.8848 -232.27
E0620 16:54:32.689153 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.690166 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.692162 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.693164 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.693164 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.694167 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.694167 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.695180 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.695180 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.695180 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.696173 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.696173 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.696173 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.697175 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.697175 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.698179 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.698179 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.698179 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.699182 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.700184 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.700184 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.700184 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.700184 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.701186 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.701186 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.701186 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.701186 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.702188 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.702188 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.702188 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.702188 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.703192 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.703192 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.704195 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.704195 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.704195 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.704195 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.705196 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.705196 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.705196 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.705700 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.705700 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.706202 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.706202 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.706702 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.706702 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.707203 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.707203 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.707705 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.709208 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.709710 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.710211 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.710211 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.710713 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.710713 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.711215 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.711215 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.711716 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.711716 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.712218 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.712218 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.712723 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.713219 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.713219 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.713721 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.713721 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.714222 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.714222 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.714723 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.714723 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.715225 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.715225 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.715726 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.715726 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.716228 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.716228 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.716729 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.717229 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.717229 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.717731 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.717731 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.719736 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.719736 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.720238 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.720238 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.720238 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.720238 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.721310 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.721310 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.721310 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.722312 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.722312 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.722312 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.722312 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.723316 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.724318 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.724318 12744 levenberg_marquardt.h:138] Failed to solve
E0620 16:54:32.724318 12744 levenberg_marquardt.h:138] Failed to solve
No message for me. SFM Dependancy version are :
No preference for use of exported gflags CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported gflags CMake configuration if available. Found installed version of gflags: C:/Program Files/gflags/CMake Detected gflags version: 2.2.0 Found required Ceres dependency: Eigen version 3.3.0 in G:/Lib/eigen-eigen No preference for use of exported glog CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported glog CMake configuration if available. Found installed version of glog: F:/lib/glog Detected glog version: 0.3.4 Found required Ceres dependency: Glog in Found Ceres version: 1.12.0 installed in: C:/Program Files/Ceres with components: [LAPACK, SchurSpecializations, OpenMP] Checking SFM deps... TRUE
C:\Users\Laurent.PC-LAURENT-VISI>G:\Lib\opencv\dll2015\bin\Debug\sfm-example-match_reconstruct.exe
Reconstruction:
Estimated 3D points: 108 Estimated cameras: 4 Refined intrinsics: [1428, 0, 640; 0, 1428, 480; 0, 0, 1]
3D Visualization:
Recovering points ... [DONE] Recovering cameras ... [DONE] Rendering points ... [DONE] Rendering Cameras ... [DONE]
Press 'q' to close each windows ...
Okay, the difference seems to be the Ceres build:
-- No preference for use of exported gflags CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported gflags CMake configuration if available.
-- Found installed version of gflags: C:/Program Files/gflags/CMake
-- Detected gflags version: 2.2.0
-- Found required Ceres dependency: Eigen version 3.3.2 in C:/DEVELOPMENT/3rdparty.git/Eigen/3.3.2
-- Found required Ceres dependency: glog
-- Found installed version of gflags: C:/Program Files/gflags/CMake
-- Detected gflags version: 2.2.0
-- Found required Ceres dependency: gflags
-- Found Ceres version: 1.12.0 installed in: C:/DEVELOPMENT/3rdparty.git/Ceres/1.12 with components: [CXSparse, SparseLinearAlgebraLibrary, SchurSpecializations, OpenMP]
-- Checking SFM deps... TRUE
Yours is
Ceres with components: [LAPACK, SchurSpecializations, OpenMP]
and mine
Ceres/1.12 with components: [CXSparse, SparseLinearAlgebraLibrary, SchurSpecializations, OpenMP]
Do you have better experience with LAPACK?
I have got Intel® Math Kernel Library 2017 Update 2 for Windows* Downloaded (it is free for personnal use) Lapack is inside In opencv cmake I have :
LAPACK(MKL): LAPACK_LIBRARIES: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries/windows/mkl/lib/intel64/mkl_intel_lp64.lib;C:/Program Files (x86)/IntelSWTools/compilers_and_libraries/windows/mkl/lib/intel64/mkl_sequential.lib;C:/Program Files (x86)/IntelSWTools/compilers_and_libraries/windows/mkl/lib/intel64/mkl_core.lib
LAPACK(MKL): Support is enabled.
and in ceres-solver cmake output
> Detected Ceres version: 1.12.0 from G:/Lib/ceres-solver/include/ceres/version.h
> -- Found Eigen version 3.3.0: G:/Lib/eigen-eigen
>
> ===============================================================
> Disabling the use of Eigen as a sparse linear algebra library.
> This does not affect the covariance estimation algorithm
> which can still use the EIGEN_SPARSE_QR algorithm.
> ===============================================================
>
> A library with BLAS API found.
> -- Found LAPACK library: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2017.0.109/windows/mkl/lib/intel64_win/mkl_intel_ilp64_dll.lib;C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2017.0.109/windows/mkl/lib/intel64_win/mkl_sequential_dll.lib;C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2017.0.109/windows/mkl/lib/intel64_win/mkl_core_dll.lib
> -- Found BLAS library: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2017.0.109/windows/mkl/lib/intel64_win/mkl_intel_ilp64_dll.lib;C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2017.0.109/windows/mkl/lib/intel64_win/mkl_sequential_dll.lib;C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2017.0.109/windows/mkl/lib/intel64_win/mkl_core_dll.lib
> -- Building without SuiteSparse.
> -- Building without CXSparse.
> ===============================================================
> Compiling without any sparse library: SuiteSparse, CXSparse
> & Eigen (Sparse) are all disabled or unavailable. No sparse
> linear solvers (SPARSE_NORMAL_CHOLESKY & SPARSE_SCHUR)
> will be available when Ceres is used.
> ===============================================================
> No preference for use of exported gflags CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported gflags CMake configuration if available.
> Found installed version of gflags: C:/Program Files/gflags/CMake
> Detected gflags version: 2.2.0
> -- Found Google Flags header in: C:/Program Files/gflags/Include, in namespace: google
> No preference for use of exported glog CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported glog CMake configuration if available.
> Found installed version of glog: F:/lib/glog
> Detected glog version: 0.3.4
> -- Building with OpenMP.
> Looking for C++ include unordered_map
> Looking for C++ include unordered_map - found
> Performing Test HAVE_UNORDERED_MAP_IN_STD_NAMESPACE
> Performing Test HAVE_UNORDERED_MAP_IN_STD_NAMESPACE - Success
> -- Found unordered_map/set in std namespace.
> Looking for C++ include memory
> Looking for C++ include memory - found
> Performing Test HAVE_SHARED_PTR_IN_STD_NAMESPACE
> Performing Test HAVE_SHARED_PTR_IN_STD_NAMESPACE - Success
> -- Found shared_ptr in std namespace using <memory> header.
> -- Building Ceres as a static library.
> Enabling CERES_NO_SUITESPARSE in Ceres config.h
> Enabling CERES_NO_CXSPARSE in Ceres config.h
> Enabling CERES_USE_OPENMP in Ceres config.h
> Enabling CERES_STD_UNORDERED_MAP in Ceres config.h
> -- Build the examples.
> Configuring done
> Generating done
Opencv 3.4.13 Windows 10 VS 2017 VTK 8.2.0 I also get this problem in this example. I can see some points in the scene,and can turn around or zoom ,but I can not turn to the camera view in sample [scenereconstruction ] And i dont change any code in the example. It seems that the code "window.setViewerPose(path[0])" dont work. And i think maybe it is a bug in VTK or it is the vtk verison's problem. Have someone solved this problem? please give me a hand.
Thanks for your answer.
