Getting T265 fisheye image?

[triplehoon]

Hi. I am trying to use RTAPMAP C++ with T265 + D400 device.

And I use rtabmap::CameraRealSense2 set as dual mode.

And I wanna use T265 Fisheye camera frame, but the only I can takeImage() from D455.

Is there any way to get T265 fisheye camera image?

[sumitsarkar1]

in dual mode T265 provides odometry and D400 provides point cloud...based on odometry the point cloud of D400 get stitched...If you can point out what exactly you want to use the T265 camera frame for may be it will be more helpful..

[triplehoon]

@sumitsarkar1 I just need to get a T265 fisheye camera image while I using Rtabmap. T265 image is not for any SLAM algorithm, just want it to show up

[sumitsarkar1]

I guess on ROS platform you can do...just view the topic published in rviz...if your are using standalone RtabMap then just check insideCameraRealSense2.cpp if its possible

[triplehoon]

@sumitsarkar1 Thanks for your advice about using ROS. But I am using standalone RtabMap, because it is very hard to use ROS with my Windows-based program. So I've checked CameraRealSense2.cpp.

/// CameraRealSense2.cpp line 1427, captureImage() func which is called by takeImage() func
else if(is_right_fisheye_arrived)
				{
					cv::Mat left,right;
					if(rectifyImages_ && stereoModel_.left().isValidForRectification() && stereoModel_.right().isValidForRectification())
					{
						left = stereoModel_.left().rectifyImage(cv::Mat(rgbBuffer_.size(), rgbBuffer_.type(), (void*)rgb_frame.get_data()));
						right = stereoModel_.right().rectifyImage(cv::Mat(depthBuffer_.size(), depthBuffer_.type(), (void*)depth_frame.get_data()));
					}
					else
					{
						left = cv::Mat(rgbBuffer_.size(), rgbBuffer_.type(), (void*)rgb_frame.get_data()).clone();
						right = cv::Mat(depthBuffer_.size(), depthBuffer_.type(), (void*)depth_frame.get_data()).clone();
					}

					if(stereoModel_.left().imageHeight() == 0 || stereoModel_.left().imageWidth() == 0)
					{
						stereoModel_.setImageSize(left.size());
					}

					data = SensorData(left, right, stereoModel_, this->getNextSeqID(), stamp);
				}

I see the data is kind of getting T265 image, but I can't get SensorData from T265 when I use takeImage() function from the Camera.

I can change source code, and re-build Rtabmap, but I think It is kind of a bad turnaround.

[sumitsarkar1]

try to do cv::imshow("fisheye",left)

[matlabbe]

To get fisheye images from T265 with CameraRealsense2, you should use realsense sdk <=2.42 or fix it in latest realsense release.

You would need to calibrate T265 to be used by rtabmap, this can be done following those instructions: http://official-rtab-map-forum.206.s1.nabble.com/Slam-using-Intel-RealSense-tracking-camera-T265-tp6333p6336.html

[triplehoon]

@matlabbe Thanks for your comments. I calibrated T265 and set SDK <= 2.42.

But I can't access to T265 sensor.

My code is

	rtabmap::CameraRealSense2* camera = new rtabmap::CameraRealSense2;
	camera->setResolution(1280,720);
	
	const rtabmap::Transform extrinsics(0.006f, 0.0025f, 0.0f, 0.0f, 0.0f, 0.0f);
	camera->setOdomProvided(true);
	camera->setDualMode(true, extrinsics);
	if (!camera->init(".","943222111057")) //943222111057: T265 serial number
	{
		std::cout << "Setting error \n";
	}
	if (camera->stereoModel_.left().isFisheye())
	{
		std::cout << "Calibrated!" << std::endl;
	}
	

But I get false value for stereoModel_.left().isFisheye(). Also, I can't get fish-eyed images when I checked breakpoint at CameraRealSense2.cpp

///  CameraRealSense2.cpp / SensorData CameraRealSense2::captureImage(CameraInfo * info) / line 1408
	else if(is_left_fisheye_arrived)
			{
				if(odometryOnlyLeftStream_) // <<< set break point here
				{

[matlabbe]

You should not set dual mode, this makes ignoring T265 image data, only odometry is kept.