MIPI CSI Camera image is upside down

[lukvacek]

The camera image being published to ROS by jetbot_camera.cpp is upside-down. When reading the camera image directly with gstreamer with flip_method=0 the image the right way up, so the camera orientation on my robot is correct. Is it possible to have the image orientation published to ROS the same as gstreamer or add parameter similar to flip_method to the image publisher?

Currently have to either rotate the published image in ROS or rotate the camera when running ROS vs. non-ROS code.

[ZeppLu]

Came across the same issue, you can check my fork

[abulnk]

I am wondering about this also. If you know the solution, I will appreciate if you can share.

Thanks.

[abulnk]

Came across the same issue, you can check my fork

Hi ZeppLu,

When I used catkin_make after using the code in your fork for jetbot_camera.cpp

I get these errors:

[ 6%] Building CXX object jetbot_ros/CMakeFiles/jetbot_camera.dir/src/jetbot_camera.cpp.o /home/jetbot/workspace/catkin_ws/src/jetbot_ros/src/jetbot_camera.cpp: In function ‘bool aquireFrame()’: /home/jetbot/workspace/catkin_ws/src/jetbot_ros/src/jetbot_camera.cpp:77:40: error: ‘class gstCamera’ has no member named ‘GetResource’ static const char* frame_id = camera->GetResource().c_str(); ^~~~~~~~~~~ /home/jetbot/workspace/catkin_ws/src/jetbot_ros/src/jetbot_camera.cpp: In function ‘int main(int, char**)’: /home/jetbot/workspace/catkin_ws/src/jetbot_ros/src/jetbot_camera.cpp:117:2: error: ‘videoOptions’ was not declared in this scope videoOptions opt; ^~~~~~~~~~~~ /home/jetbot/workspace/catkin_ws/src/jetbot_ros/src/jetbot_camera.cpp:119:2: error: ‘opt’ was not declared in this scope opt.resource = camera_device; ^~~ /home/jetbot/workspace/catkin_ws/src/jetbot_ros/src/jetbot_camera.cpp:123:15: error: ‘videoOptions’ is not a class, namespace, or enumeration opt.ioType = videoOptions::INPUT; ^~~~~~~~~~~~ /home/jetbot/workspace/catkin_ws/src/jetbot_ros/src/jetbot_camera.cpp:124:19: error: ‘videoOptions’ is not a class, namespace, or enumeration opt.flipMethod = videoOptions::FLIP_ROTATE_180; ^~~~~~~~~~~~ jetbot_ros/CMakeFiles/jetbot_camera.dir/build.make:62: recipe for target 'jetbot_ros/CMakeFiles/jetbot_camera.dir/src/jetbot_camera.cpp.o' failed make[2]: *** [jetbot_ros/CMakeFiles/jetbot_camera.dir/src/jetbot_camera.cpp.o] Error 1 CMakeFiles/Makefile2:615: recipe for target 'jetbot_ros/CMakeFiles/jetbot_camera.dir/all' failed make[1]: *** [jetbot_ros/CMakeFiles/jetbot_camera.dir/all] Error 2 Makefile:140: recipe for target 'all' failed make: *** [all] Error 2 Invoking "make -j4 -l4" failed

I updated the jetson-utils

It did not help. Please let me know if you have any insights.

[DavidCziu]

/*

• Copyright (c) 2019, NVIDIA CORPORATION. All rights reserved.
• Permission is hereby granted, free of charge, to any person obtaining a
• copy of this software and associated documentation files (the "Software"),
• to deal in the Software without restriction, including without limitation
• the rights to use, copy, modify, merge, publish, distribute, sublicense,
• and/or sell copies of the Software, and to permit persons to whom the
• Software is furnished to do so, subject to the following conditions:
• The above copyright notice and this permission notice shall be included in
• all copies or substantial portions of the Software.
• THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
• IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
• FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
• THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
• LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
• FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
• DEALINGS IN THE SOFTWARE. */

#include <ros/ros.h>

#include <sensor_msgs/Image.h> #include <sensor_msgs/image_encodings.h>

#include <image_transport/image_transport.h>

#include <jetson-utils/gstCamera.h> #include <jetson-utils/cudaNormalize.h>

#include "image_converter.h"

// globals gstCamera* camera = NULL;

imageConverter* camera_cvt = NULL; ros::Publisher* camera_pub = NULL;

// aquire and publish camera frame bool aquireFrame() { float4* imgRGBA = NULL;

// get the latest frame
if( !camera->CaptureRGBA((float**)&imgRGBA, 1000) )
{
	ROS_ERROR("failed to capture camera frame");
	return false;
}

// assure correct image size
if( !camera_cvt->Resize(camera->GetWidth(), camera->GetHeight(), IMAGE_RGBA32F) )
{
	ROS_ERROR("failed to resize camera image converter");
	return false;
}

// populate the message
sensor_msgs::Image msg;

if( !camera_cvt->Convert(msg, imageConverter::ROSOutputFormat, imgRGBA) )
{
	ROS_ERROR("failed to convert camera frame to sensor_msgs::Image");
	return false;
}

// publish the message
camera_pub->publish(msg);
ROS_INFO("published camera frame");
return true;

}

// node main loop int main(int argc, char **argv) { ros::init(argc, argv, "jetbot_camera");

ros::NodeHandle nh;
ros::NodeHandle private_nh("~");

/*
 * retrieve parameters
 */
std::string camera_device = "0";	// MIPI CSI camera by default
// width and height should be of uint32_t, but XML supports signed integers only
int width = 224, height = 224;	

//int width = 1280, height = 720; float framerate = 30.0;

private_nh.param<std::string>("device", camera_device, camera_device);
private_nh.param("width", width, width);
private_nh.param("height", height, height);
private_nh.param("framerate", framerate, framerate);

ROS_INFO("opening camera device %s @ %dx%d %ffps", camera_device.c_str(), width, height, framerate);

//ROS_INFO("opening camera device %s", camera_device.c_str());


/*
 * open camera device
 */

videoOptions opt;
opt.resource = camera_device;
opt.width = width;
opt.height = height;
opt.frameRate = framerate;
opt.ioType = videoOptions::INPUT;
opt.flipMethod = videoOptions::FLIP_ROTATE_180;

camera = gstCamera::Create(opt);
//camera = gstCamera::Create(camera_device.c_str());

if( !camera )
{
	ROS_ERROR("failed to open camera device %s", camera_device.c_str());
	return 0;
}


/*
 * create image converter
 */
camera_cvt = new imageConverter();

if( !camera_cvt )
{
	ROS_ERROR("failed to create imageConverter");
	return 0;
}


/*
 * advertise publisher topics
 */
image_transport::ImageTransport it(private_nh);
ros::Publisher camera_publisher = private_nh.advertise<sensor_msgs::Image>("raw", 2);
camera_pub = &camera_publisher;


/*
 * start the camera streaming
 */
if( !camera->Open() )
{
	ROS_ERROR("failed to start camera streaming");
	return 0;
}


/*
 * start publishing video frames
 */
while( ros::ok() )
{
	//if( raw_pub->getNumSubscribers() > 0 )
		aquireFrame();

	ros::spinOnce();
}

delete camera;
return 0;

}

this worked for me. Don't forget to also edit your CMakeLists.txt. I had to add image_transport to find package in CMakeList.txt and also <build_depend>image_transport</build_depend> and <exec_depend>image_transport</exec_depend> to package.xml

[abulnk]

David,

Thank you. It worked. For others reading this, here are the details.

I used the code sent by David to replace jetbot_camera.cpp in ~/workspace/catkin_ws/src/jetbot_ros/src

except I used (as a personal choice)

int width = 640, height = 360;

In CMakeLists.txt, I added

find_package(catkin REQUIRED COMPONENTS roscpp rospy std_msgs sensor_msgs image_transport )

In package.xml, I added

<buildtool_depend>catkin</buildtool_depend> <build_depend>roscpp</build_depend> <build_depend>rospy</build_depend> <build_depend>std_msgs</build_depend> <build_depend>image_transport</build_depend> <build_export_depend>roscpp</build_export_depend> <build_export_depend>rospy</build_export_depend> <build_export_depend>std_msgs</build_export_depend> <exec_depend>roscpp</exec_depend> <exec_depend>rospy</exec_depend> <exec_depend>std_msgs</exec_depend> <exec_depend>image_transport</exec_depend>