Camera spec

[antithing]

Hi again, not an issue, just a question... I am going to update my stereo camera setup to run your system live. Is 640 x 480 enough? Or would you recommend a higher resolution?

Global shutter, 120 fps...

Thanks again for your code!

[Beewe]

No this resolution should be fine, in all my tests I use exactly this resolution, but I have no experience in stereo camera setup / stereo SLAM. Note: With increasing resolution the tracking process needs more time per frame, so unless you have a very powerfull computer I would suggest not to increase the resolution, as the amount of tracked frames per second is more important in most cases (especially in a live system). You're welcome!

[antithing]

Great, thank you!

On Mon, Feb 1, 2016 at 2:57 PM, Benjamin [email protected] wrote:

No this resolution should be fine, all my tests I do use exactly this resolution, but I have no experience in stereo camera setup / stereo SLAM. Note: With increasing resolution the tracking process needs more time per frame, so unless you have a very powerfull computer I would suggest not to increase the resolution, as the amount of tracked frames per second is more important in most cases (especially in a live system). You're welcome!

— Reply to this email directly or view it on GitHub https://github.com/Beewe/ORB_SLAM2_Windows/issues/1#issuecomment-178003713 .

[Beewe]

Much easier ;) the Tracking-Method already returns the camera pose. so the only thing you need to add to your example is: cv::Mat cameraPose = SLAM.TrackMonocular(im,tframe);

[Beewe]

The camera pose you get is indeed not in world coordinates, you will need the inverse. So you basically get [R T;0 1] and the inverse would be [R^t -R^t*T;0 1]. Have a deeper look at the code, in MapDrawer::GetCurrentOpenGLCameraMatrix() you'll find everything you need. Since there is no measurement about absolute scale and no information about the real world, it's common to set the first camera pose to the origin of the world. All further poses are then estimated relative to this first pose.

[Beewe]

Camera.bf divided by Camera.fx results in the baseline in meters. So 386.1448/718.856 = 0.5372m.

[Beewe]

Yes it sounds like some image conversion problem, but i guess there is no ways getting around debugging the code and look for the line number where the crash occurs.

[meiroo]

hi, Beewe. First thank you for your code. May I ask some questions also about the camera pose ?

cv::Mat cameraPose = SLAM.TrackMonocular(im,tframe);

I know I can get the camera pose like this. I have tested use the camera pose to draw the map points.

            const vector<ORB_SLAM2::MapPoint*> &vpMPs = map->GetAllMapPoints();
            std::vector<cv::Point3f> allmappoints;
            ... ... 
            std::vector<cv::Point3f> maptrans;
            float scale = 1.0f;
            cv::perspectiveTransform(allmappoints, maptrans, pose); 
            for (size_t j = 0; j < maptrans.size(); ++j) {
                cv::Point3f r1 = maptrans[j];
                r1.x = (r1.x+1)*320;
                r1.y = (r1.y+1)*240;
                cv::rectangle(im, cv::Rect((r1.x - 5) * scale, (r1.y - 5 )* scale, 5 * 2 *scale, 5 * 2 * scale), cv::Scalar( 0, 255, 0 ),1);
            }

As above code, I use the camera pose matrix to transform the map points, and I get the transformed point in range (-1,-1,-1 ) to (1,1,1). And then I use (r1.x+1) * 320 , (r1.y+1) * 240 to calculate the map point position in the screen ( 640 x 480 ).

But the result is not very right. When I move the camera up, the map points drawn in the screen also moved above, the scale is also seemed wrong. I also try to use MapDrawer::GetCurrentOpenGLCameraMatrix() to modify the pose matrix then do the perspectiveTransform but get same results.

Is there something I missed ? How can I use the camera pose to draw the map points and make them exactly match the screen image ?

[Beewe]

You'll need cv::projectPoints instead of cv::perspectiveTransform. Then you need to get mK (camera matrix) and mDistCoef (distortion coefficients) from the Tracker instance. The pose must be splitted into rotation and translation vector. Quick example code follows:

cv::Mat rVec;
cv::Rodrigues(pose.colRange(0, 3).rowRange(0, 3), rVec);
cv::Mat tVec = pose.col(3).rowRange(0, 3);

const vector<ORB_SLAM2::MapPoint*> vpMPs = map->GetAllMapPoints(); 
if (vpMPs.size() > 0) {
   std::vector<cv::Point3f> allmappoints;
   for (size_t i = 0; i < vpMPs.size(); i++) {
      if (vpMPs[i]) {
         cv::Point3f pos = cv::Point3f(vpMPs[i]->GetWorldPos());
         allmappoints.push_back(pos);
      }
   }
   std::vector<cv::Point2f> projectedPoints;
   cv::projectPoints(allmappoints, rVec, tVec, mK, mDistCoef, projectedPoints);
   for (size_t j = 0; j < projectedPoints.size(); ++j) {
      cv::Point2f r1 = projectedPoints[j];
      cv::circle(im, cv::Point(r1.x, r1.y), 2, cv::Scalar(0, 255, 0), 1, 8);
   }
}

[meiroo]

@Beewe Thanks so much! I'm confusing about this for nearly two days. And your answer is perfectly solved this problem.