camera_calibration
camera_calibration copied to clipboard
puzzlepaint
Having trouble with running calibration script
Hello, I am able to successfully run the feature detection for 2 cameras I have
(lab42) root@gpu2:/nfs1/mostafa/camera_calibration/build# ./applications/camera_calibration/camera_calibration --pattern_files /nfs1/mostafa/custom_pattern.yaml --image_directories /nfs1/mostafa/cali$
ration_images/4103636377,/nfs1/mostafa/calibration_images/4103642089 --dataset_output_path /nfs1/mostafa/fea
tures_10px.bin --refinement_window_half_extent 10
time file:line v|
23:27:47.735 loguru.cpp:587 INFO| arguments: ./applications/camera_calibration/camera_calibration --pattern_files /nfs1/mostafa/custom_pattern.yaml --image_directories /nfs1/mostafa/calibratio$
_images/4103636377,/nfs1/mostafa/calibration_images/4103642089 --dataset_output_path /nfs1/mostafa/features_10px.bin --refinement_window_half_extent 10
23:27:47.735 loguru.cpp:590 INFO| Current dir: /nfs1/mostafa/camera_calibration/build
23:27:47.735 loguru.cpp:592 INFO| stderr verbosity: 2
23:27:47.735 loguru.cpp:593 INFO| -----------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
23:27:48.539 calibration.cc:1175 INFO| Found 15 images to extract features from.
23:27:48.786 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/0.png: 52 features
23:27:48.824 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/1.png: 52 features
23:27:48.868 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/10.png: 50 features
23:27:48.911 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/11.png: 52 features
23:27:48.954 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/12.png: 52 features
23:27:49.002 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/13.png: 52 features
23:27:49.052 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/14.png: 52 features
23:27:49.097 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/2.png: 50 features
23:27:49.138 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/3.png: 52 features
23:27:49.174 feature_detector_tagged:729 INFO| Timing
------
DetectFeatures() 10 000000.4291 (00.042912 +- 00.060850) [00.020548,00.216021]
AprilTag detection 10 000000.1098 (00.010982 +- 00.001589) [00.008832,00.013215]
PredictAndDetectFeatures() 10 000000.1045 (00.010452 +- 00.001626) [00.008745,00.013060]
RefineFeatureDetections() 48 000000.0883 (00.001841 +- 00.000373) [00.000539,00.002531]
23:27:49.174 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/4.png: 46 features
23:27:49.218 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/5.png: 52 features
23:27:49.257 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/6.png: 21 features
23:27:49.294 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/7.png: 51 features
23:27:49.340 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/8.png: 52 features
23:27:49.388 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103636377/9.png: 50 features
23:27:49.901 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/0.png: 51 features
23:27:50.199 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/1.png: 50 features
23:27:50.347 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/10.png: 52 features
23:27:50.625 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/11.png: 50 features
23:27:50.833 feature_detector_tagged:729 INFO| Timing
------
DetectFeatures() 20 000000.6796 (00.033978 +- 00.042980) [00.020548,00.216021]
AprilTag detection 20 000000.2188 (00.010939 +- 00.002025) [00.008742,00.015323]
PredictAndDetectFeatures() 20 000000.2096 (00.010478 +- 00.001681) [00.008584,00.013676]
RefineFeatureDetections() 94 000000.1740 (00.001852 +- 00.000359) [00.000539,00.002600]
23:27:50.833 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/12.png: 52 features
23:27:50.969 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/13.png: 52 features
23:27:51.172 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/14.png: 52 features
23:27:51.268 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/2.png: 43 features
23:27:51.498 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/3.png: 51 features
23:27:51.780 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/4.png: 49 features
23:27:51.918 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/5.png: 49 features
23:27:52.049 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/6.png: 39 features
23:27:52.207 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/7.png: 50 features
23:27:52.403 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/8.png: 51 features
23:27:52.808 feature_detector_tagged:729 INFO| Timing
------
DetectFeatures() 30 000000.9500 (00.031666 +- 00.034993) [00.020548,00.216021]
AprilTag detection 30 000000.3433 (00.011443 +- 00.002303) [00.008742,00.017321]
PredictAndDetectFeatures() 30 000000.3172 (00.010573 +- 00.001670) [00.008584,00.013676]
RefineFeatureDetections() 143 000000.2658 (00.001859 +- 00.000379) [00.000539,00.002633]
23:27:52.808 calibration.cc:1210 INFO| /nfs1/mostafa/calibration_images/4103642089/9.png: 52 features
23:27:52.843 loguru.cpp:467 INFO| atexit
however, when i actually try to run the calibration I get this issue,
lab42) root@gpu2:/nfs1/mostafa/camera_calibration/build# ./applications/camera_calibration/camera_calibration --dataset_files /nfs1/mostafa/features_10px.bin --output_directory /nfs1/mostafa/result_10px_central_opencv_50 --cell_length_in_pixels 50 --model central_generic --num_pyramid_levels 4
time file:line v|
00:49:50.136 loguru.cpp:587 INFO| arguments: ./applications/camera_calibration/camera_calibration --dataset_files /nfs1/mostafa/features_10px.bin --output_directory /nfs1/mostafa/result_10px_central_opencv_50 --cell_length_in_pixels 50 --model central_generic --num_pyramid_levels 4
00:49:50.136 loguru.cpp:590 INFO| Current dir: /nfs1/mostafa/camera_calibration/build
00:49:50.136 loguru.cpp:592 INFO| stderr verbosity: 2
00:49:50.136 loguru.cpp:593 INFO| -----------------------------------
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
00:49:50.980 calibration.cc:1268 INFO| Dataset 0: /nfs1/mostafa/features_10px.bin
00:49:50.981 calibration_io.cc:244 INFO| Loaded dataset with 1479 feature observations.
00:49:50.981 calibration.cc:937 INFO| Calibrate() starting ...
00:49:50.981 calibration.cc:951 INFO| Estimating the dense initialization for camera 0 ...
00:49:54.875 dense_initialization.cc:1310 INFO| Final init_indices: 12, 7, 1
00:49:54.875 dense_initialization.cc:1311 INFO| best_num_point_triples: 3757
00:49:54.875 dense_initialization.cc:997 INFO| optical_center: -0.0761606 0.166054 -1.52539
00:49:54.875 dense_initialization.cc:998 INFO| init_known_geometry_index: 0
00:49:54.875 dense_initialization.cc:1002 INFO| Determining initial partial calibration from initial images ...
00:49:54.878 dense_initialization.cc:1040 INFO| Extending initial calibration by localizing the remaining images ...
double free or corruption (out)
Aborted
Do you have any idea what is going on or what I should do?
To pin down the issue, it would be helpful to run the application in gdb and/or valgrind (while ensuring that it is built in either RelWithDebInfo or Debug mode). For example, run:
gdb --ex run --args ./applications/camera_calibration/camera_calibration --dataset_files /nfs1/mostafa/features_10px.bin --output_directory /nfs1/mostafa/result_10px_central_opencv_50 --cell_length_in_pixels 50 --model central_generic --num_pyramid_levels 4
Once it crashes, type bt and press return to get a backtrace.
Hey, I just got the same "double free or corruption" trouble. And I tried to use gdb and it showed:
By the way, when it started calibrating, I got warnings said:
WARNING: Negative value in sqrt(): -20.0058
Many warnings appeared with different negative numbers, I'm not sure whether it matters.
Thanks for the gdb output. I had similar issues to this before when the compile flags between my application and OpenGV differed. I suspect that this might lead to a different memory layout for objects from the Eigen library, which then causes invalid accesses.
Is there any chance that this might be the case in your environment? In case you compiled OpenGV yourself with default settings, it should work, I believe. But in case it has been configured differently, e.g., as part of a system-installed package, maybe it could fail due to such an inconsistency.
The relevant compiler flag was -march=native. By default this should be set by both the camera calibration application and in OpenGV. You could try removing this flag for the camera calibration application (in its CMakeLists.txt files, both in the root directory of the sources and in applications/camera_calibration/CMakeLists.txt in all places where it appears) and see whether it helps.
Regarding the negative values in sqrt, I have seen those as well but I don't know what exactly causes them. This indicates some error in the initialization part, however, this should usually not matter, since the initialization is attempted many times with different inputs, and the best result is chosen. So it should not matter if it fails a few times.
Thanks for your advice. I tried remove all -march=native flags in both CMakeLists.txt files. But it turned out another error (still related to memory problem, I guess):
Okay, seems like the flag might not be the problem. But I believe that it is likely to be some inconsistency of that kind. For example, another possibility would be that OpenGV and the camera calibration application could be compiled against different versions of Eigen. But I can only guess the exact reason.
(By the way, it is possible to paste command-line output as text here, preferably delimited by ``` for nice formatting; there is no need to attach it as images.)
Sorry about the images, I have to do so because I am working on a remote server with no display while the application requires GUI. I hate those images too😷.
So next step should I check the exact version of OpenGV and Eigen?
That is at least what I would try to check if I got these errors. All used library versions and compiler flags should be identical for compiling OpenGV and for compiling the camera calibration application, otherwise there might be the possibility of incompatible differences.
Another thing to try is to run the OpenGV test that is among the unit tests. If that crashes, it is a pretty clear sign that there is indeed some kind of inconsistency in the compiling that causes the error. If it does not crash, maybe the error is somewhere else. It should be possible to run this test as follows after compiling camera_calibration_test:
./applications/camera_calibration/camera_calibration_test --gtest_filter=OpenGV.AbsolutePoseSacProblem
hi, @puzzlepaint : I got the same problem like :
*** Error in `/home/lingbo/cam-calib-ws/camera_calibration/build/applications/camera_calibration/camera_calibration': double free or corruption (out): 0x00007fffd0001600 ***
......
Thread 4 "camera_calibrat" received signal SIGABRT, Aborted. [Switching to Thread 0x7fffe13bd700 (LWP 28954)] 0x00007ffff514b428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
when I run with:
sudo gdb --ex run --args ./camera_calibration/build/applications/camera_calibration/camera_calibration --dataset_files ./features_15px.bin --output_directory ./result_15px_central_opencv_50 --cell_length_in_pixels 50 --model central_generic --num_pyramid_levels 4 --show_visualizations
and I type 'bt':
Thread 4 "camera_calibrat" received signal SIGABRT, Aborted. [Switching to Thread 0x7fffe13bd700 (LWP 31497)] 0x00007ffff514b428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 54 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 0x00007ffff514b428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54 #1 0x00007ffff514d02a in __GI_abort () at abort.c:89 #2 0x00007ffff518d7ea in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7ffff52a6ed8 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:175 #3 0x00007ffff519637a in malloc_printerr (ar_ptr=
, ptr= , str=0x7ffff52a6fe8 "double free or corruption (out)", action=3) at malloc.c:5006 #4 _int_free (av= , p= , have_lock=0) at malloc.c:3867 #5 0x00007ffff519a53c in __GI___libc_free (mem= ) at malloc.c:2968 #6 0x00007ffff782eb38 in opengv::absolute_pose::modules::p3p_kneip_main(std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, Eigen::aligned_allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > const&, std::vector<Eigen::Matrix<double, 3, 1, 0, 3, 1>, Eigen::aligned_allocator<Eigen::Matrix<double, 3, 1, 0, 3, 1> > > const&, std::vector<Eigen::Matrix<double, 3, 4, 0, 3, 4>, Eigen::aligned_allocator<Eigen::Matrix<double, 3, 4, 0, 3, 4> > >&) () from /home/tusimple/cam-calib-ws/camera_calibration/build/applications/camera_calibration/libcamera_calibration_baselib.so #7 0x00007ffff78135cb in opengv::absolute_pose::p3p_kneip(opengv::absolute_pose::AbsoluteAdapterBase const&, unsigned long, unsigned long, unsigned long) () from /home/tusimple/cam-calib-ws/camera_calibration/build/applications/camera_calibration/libcamera_calibration_baselib.so #8 0x00007ffff781368e in opengv::absolute_pose::p3p_kneip(opengv::absolute_pose::AbsoluteAdapterBase const&, std::vector<int, std::allocator > const&) () from /home/tusimple/cam-calib-ws/camera_calibration/build/applications/camera_calibration/libcamera_calibration_baselib.so #9 0x00007ffff7828e91 in opengv::sac_problems::absolute_pose::AbsolutePoseSacProblem::computeModelCoefficients(std::vector<int, std::allocator > const&, Eigen::Matrix<double, 3, 4, 0, 3, 4>&) const () from /home/tusimple/cam-calib-ws/camera_calibration/build/applications/camera_calibration/libcamera_calibration_baselib.so #10 0x00007ffff76fc950 in opengv::sac::Ransacopengv::sac_problems::absolute_pose::AbsolutePoseSacProblem::computeModel (this=this@entry=0x7fffe13b92e0, debug_verbosity_level=debug_verbosity_level@entry=0) at /home/tusimple/opengv/include/opengv/sac/implementation/Ransac.hpp:78 #11 0x00007ffff76f55ef in vis::LocalizePattern (calibration_sum=..., calibration_count=..., pattern=std::vector of length 253, capacity 256 = {...}, features=std::vector of length 253, capacity 256 = {...}, first_feature=first_feature@entry=0, last_feature= , min_calibrated_match_count=7, out_pattern_tr_image=0x7fffe13b9e80) at /home/tusimple/cam-calib-ws/camera_calibration/applications/camera_calibration/src/camera_calibration/calibration_initialization/dense_initialization.cc:392 #12 0x00007ffff76f84cb in vis::DenseInitialization::AttemptToLocalizeImage (this=this@entry=0x7fffe13ba600, p=..., imageset_index=imageset_index@entry=0, calibration_sum=..., calibration_count=..., global_tr_image=0x7fffe13b9e80, dense_matches=0x7fffe13b9cf0) at /home/tusimple/cam-calib-ws/camera_calibration/applications/camera_calibration/src/camera_calibration/calibration_initialization/dense_initialization.cc:1114 #13 0x00007ffff76fa24f in vis::DenseInitialization::InitializeCamera (this=this@entry=0x7fffe13ba600, dataset=dataset@entry=0x7fffe13bbc20, camera_index=camera_index@entry=0, localize_only=localize_only@entry=false, calibration_window=calibration_window@entry=0x7fffffffc700, step_by_step=step_by_step@entry=false) at /home/tusimple/cam-calib-ws/camera_calibration/applications/camera_calibration/src/camera_calibration/calibration_initialization/dense_initialization.cc:1352 #14 0x00007ffff77dc19f in vis::Calibrate (dataset=dataset@entry=0x7fffe13bbc20, dense_initialization_path=0x0, state_initialization_base_path= , outlier_visualization_path=0x6c8850 "./result_15px_central_opencv_50/report", use_cuda=use_cuda@entry=false, schur_mode=schur_mode@entry=vis::Dense, num_pyramid_levels=4, model_type=vis::CameraModel::Type::CentralGeneric, approx_pixels_per_cell=50, regularization_weight=regularization_weight@entry=0, outlier_removal_factor=outlier_removal_factor@entry=6, localize_only=false, calibration_window=0x7fffffffc700, state=0x7fffe13bbc90, dataset_output_path=0x6c8750 "./result_15px_central_opencv_50/dataset.bin", state_output_path=0x6c7ea0 "./result_15px_central_opencv_50") at /home/tusimple/cam-calib-ws/camera_calibration/applications/camera_calibration/src/camera_calibration/calibration.cc:955 #15 0x00007ffff77dd737 in vis::CalibrateBatch (image_directories=..., dataset_files=std::vector of length 1, capacity 1 = {...}, dense_initialization_base_path="", state_directory="", dataset_output_path=..., state_output_directory="./result_15px_central_opencv_50", pruned_dataset_output_path="./result_15px_central_opencv_50/dataset.bin", report_base_path="./result_15px_central_opencv_50/report", detector=0x0, num_pyramid_levels=4, model_type=vis::CameraModel::Type::CentralGeneric, cell_length_in_pixels=50, regularization_weight=0, outlier_removal_factor=6, localize_only=false, schur_mode=vis::Dense, calibration_window=0x7fffffffc700)
I am checking the version of the dependency, or is there a way to solve it ? thks
I solved this problem. After recompiling with the same version of Eigen, I solved the above problem.