isaac_ros_object_detection icon indicating copy to clipboard operation
isaac_ros_object_detection copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA-ISAAC-ROS

System throttled due to over current when running Yolov8 on Orin-NX 16GB

Open mzahana opened this issue 1 year ago • 5 comments

Greetings.

I have a Jetson Oin NX 16GB with carrier board J401 from seedstudio.

When I run isaac_ros_yolov8, I continuously get the following warning 'System throttled due to over-current.'

I am using the MAXN power mode.

Here is a log file generated by the Jetson Power GUI. yolo_over_current_issue_orin_nx16.csv

I also noticed that the cashed memory gets large when I run this package! Here is a screenshot from jtop Screenshot from 2023-11-30 08-26-27

Here is a small video that shows the output of jtop jtop1.webm

Here are the commands I run with their terminal outputs.

command

ros2 launch isaac_ros_yolov8 realsense.launch.py

realsense.launch.py

# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0

import launch
from launch_ros.actions import ComposableNodeContainer, Node
from launch_ros.descriptions import ComposableNode


def generate_launch_description():
    """Launch file which brings up visual slam node configured for RealSense."""
    realsense_camera_node = Node(
        name='camera',
        namespace='camera',
        package='realsense2_camera',
        executable='realsense2_camera_node',
        parameters=[{
                'align_depth.enable': True,
                'enable_infra1': False,
                'enable_infra2': False,
                'enable_color': True,
                'enable_depth': True,
                'depth_module.emitter_enabled': 0,
                'rgb_camera.profile': '1280x720x30', 
                'depth_module.profile': '640x480x90',
                'enable_gyro': False,
                'enable_accel': False,
                'gyro_fps': 200,
                'accel_fps': 200,
                'unite_imu_method': 2
        }],
        #remappings=[('color/image_raw', '/image')
        #]
    )
    
    image_resize_node_color = ComposableNode(
        package='isaac_ros_image_proc',
        plugin='nvidia::isaac_ros::image_proc::ResizeNode',
        name='image_resize_right',
        parameters=[{
                'output_width': 640,
                'output_height': 640,
                'keep_aspect_ratio': True
        }],
        remappings=[
            ('camera_info', '/camera/color/camera_info'),
            ('image', '/camera/color/image_raw'),
            ('resize/camera_info', '/camera/color/camera_info_resize'),
            ('resize/image', '/image')]
    )
    
    resize_launch_container = ComposableNodeContainer(
        name='resize_launch_container',
        namespace='',
        package='rclcpp_components',
        executable='component_container',
        composable_node_descriptions=[
            image_resize_node_color
        ],
        output='screen'
    )

    return launch.LaunchDescription([realsense_camera_node, resize_launch_container])
   # return launch.LaunchDescription([realsense_camera_node])

output

[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-11-30-08-39-54-
528665-ubuntu-563055
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-1]: process started with pid [563066]
[INFO] [component_container-2]: process started with pid [563068]
[component_container-2] [INFO] [1701322795.155558550] [resize_launch_container]: Load
Library: /workspaces/isaac_ros-dev/install/isaac_ros_image_proc/lib/libresize_node.so
[realsense2_camera_node-1] [INFO] [1701322795.179117503] [camera.camera]: RealSense
ROS v4.51.1
[realsense2_camera_node-1] [INFO] [1701322795.179345798] [camera.camera]: Built with
LibRealSense v2.54.2
[realsense2_camera_node-1] [INFO] [1701322795.179368646] [camera.camera]: Running with
LibRealSense v2.54.2
[component_container-2] [INFO] [1701322795.215812099] [resize_launch_container]: Found
class: rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ResizeNode>
[component_container-2] [INFO] [1701322795.215926118] [resize_launch_container]:
Instantiate class:
rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::image_proc::ResizeNode>
[component_container-2] [INFO] [1701322795.262946420] [NitrosContext]: [NitrosContext]
Creating a new shared context
[component_container-2] [INFO] [1701322795.263211420] [image_resize_right]: [NitrosNode]
Initializing NitrosNode
[component_container-2] [INFO] [1701322795.265203222] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/std/libgxf_std.so
[component_container-2] [INFO] [1701322795.283212574] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container-2] [INFO] [1701322795.287727520] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/libgxf_sight.so
[component_container-2] [INFO] [1701322795.306618338] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/libgxf_atlas.so
[component_container-2] [INFO] [1701322795.324723757] [NitrosContext]: [NitrosContext]
Loading application: '/workspaces/isaac_ros-
dev/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_grap
h.yaml'
[component_container-2] [INFO] [1701322795.326122357] [NitrosContext]: [NitrosContext]
Initializing application...

[component_container-2] [INFO] [1701322795.342824728] [NitrosContext]: [NitrosContext]
Running application...
[component_container-2] 2023-11-30 08:39:55.342 WARN gxf/std/program.cpp@514: No
system specified. Nothing to do
[component_container-2] [INFO] [1701322795.353971930] [image_resize_right]: [NitrosNode]
Starting NitrosNode
[component_container-2] [INFO] [1701322795.354067165] [image_resize_right]: [NitrosNode]
Loading built-in preset extension specs
[component_container-2] [INFO] [1701322795.357669349] [image_resize_right]: [NitrosNode]
Loading built-in extension specs
[component_container-2] [INFO] [1701322795.357795816] [image_resize_right]: [NitrosNode]
Loading preset extension specs
[component_container-2] [INFO] [1701322795.369142448] [image_resize_right]: [NitrosNode]
Loading extension specs
[component_container-2] [INFO] [1701322795.369308085] [image_resize_right]: [NitrosNode]
Loading generator rules
[component_container-2] [INFO] [1701322795.369807075] [image_resize_right]: [NitrosNode]
Loading extensions
[component_container-2] [INFO] [1701322795.372091589] [image_resize_right]:
[NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so
[component_container-2] [INFO] [1701322795.374778387] [image_resize_right]:
[NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container-2] [INFO] [1701322795.377150327] [image_resize_right]:
[NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so
[component_container-2] [INFO] [1701322795.399408506] [NitrosContext]: [NitrosContext]
Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container-2] [INFO] [1701322795.409400379] [image_resize_right]: [NitrosNode]
Loading graph to the optimizer
[component_container-2] [INFO] [1701322795.423973344] [image_resize_right]: [NitrosNode]
Running optimization
[realsense2_camera_node-1] [INFO] [1701322795.550378595] [camera.camera]: Device with
serial number 207322250752 was found.
[realsense2_camera_node-1]
[realsense2_camera_node-1] [INFO] [1701322795.550516743] [camera.camera]: Device with
physical ID 2-1.3-3 was found.
[realsense2_camera_node-1] [INFO] [1701322795.550533736] [camera.camera]: Device with
name Intel RealSense D455 was found.
[realsense2_camera_node-1] [INFO] [1701322795.551027542] [camera.camera]: Device with
port number 2-1.3 was found.
[realsense2_camera_node-1] [INFO] [1701322795.551053079] [camera.camera]: Device USB
type: 3.2
[realsense2_camera_node-1] [INFO] [1701322795.567075653] [camera.camera]:
getParameters...
[realsense2_camera_node-1] [INFO] [1701322795.573704453] [camera.camera]: JSON file is

not provided
[realsense2_camera_node-1] [INFO] [1701322795.575166287] [camera.camera]: Device
Name: Intel RealSense D455
[realsense2_camera_node-1] [INFO] [1701322795.575854211] [camera.camera]: Device Serial
No: 207322250752
[realsense2_camera_node-1] [INFO] [1701322795.576485301] [camera.camera]: Device
physical port: 2-1.3-3
[realsense2_camera_node-1] [INFO] [1701322795.576511222] [camera.camera]: Device FW
version: 5.14.0
[realsense2_camera_node-1] [INFO] [1701322795.576524630] [camera.camera]: Device
Product ID: 0x0B5C
[realsense2_camera_node-1] [INFO] [1701322795.576535831] [camera.camera]: Sync Mode:
Off
[component_container-2] [INFO] [1701322795.579101473] [image_resize_right]: [NitrosNode]
Obtaining graph IO group info from the optimizer
[component_container-2] [INFO] [1701322795.583747303] [image_resize_right]: [NitrosNode]
Creating negotiated publishers/subscribers
[component_container-2] [INFO] [1701322795.631932567] [image_resize_right]: [NitrosNode]
Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/image_resize_right' in
container '/resize_launch_container'
[component_container-2] [INFO] [1701322795.647637116] [image_resize_right]: Negotiating
[component_container-2] [INFO] [1701322795.651419818] [image_resize_right]: Negotiating
[component_container-2] [INFO] [1701322796.640272703] [image_resize_right]: [NitrosNode]
Starting post negotiation setup
[component_container-2] [INFO] [1701322796.640406851] [image_resize_right]: [NitrosNode]
Getting data format negotiation results
[component_container-2] [INFO] [1701322796.640440036] [image_resize_right]:
[NitrosPublisher] Use the negotiated data format: "nitros_image_rgb8"
[component_container-2] [INFO] [1701322796.640456644] [image_resize_right]:
[NitrosPublisher] Negotiation ended with no results
[component_container-2] [INFO] [1701322796.640469125] [image_resize_right]:
[NitrosPublisher] Use only the compatible publisher:
topic_name="/camera/color/camera_info_resize", data_format="nitros_camera_info"
[component_container-2] [INFO] [1701322796.640480805] [image_resize_right]:
[NitrosSubscriber] Negotiation ended with no results
[component_container-2] [INFO] [1701322796.640492517] [image_resize_right]:
[NitrosSubscriber] Use the compatible subscriber: topic_name="/camera/color/image_raw",
data_format="nitros_image_bgr8"
[component_container-2] [INFO] [1701322796.640597480] [image_resize_right]:
[NitrosSubscriber] Negotiation ended with no results
[component_container-2] [INFO] [1701322796.640610217] [image_resize_right]:
[NitrosSubscriber] Use the compatible subscriber: topic_name="/camera/color/camera_info",
data_format="nitros_camera_info"

[component_container-2] [INFO] [1701322796.640669962] [image_resize_right]:
[NitrosPublisherSubscriberGroup] Adjusted the compatible format of the component
"input_compositor/image_in" (type="nvidia::gxf::DoubleBufferReceiver") from
"nitros_image_bgr8" to "nitros_image_rgb8"
[component_container-2] [INFO] [1701322796.641014836] [image_resize_right]: [NitrosNode]
Exporting the final graph based on the negotiation results
[component_container-2] [INFO] [1701322796.648775253] [image_resize_right]: [NitrosNode]
Wrote the final top level YAML graph to
"/tmp/isaac_ros_nitros/graphs/JTILYGRDJC/JTILYGRDJC.yaml"
[component_container-2] [INFO] [1701322796.648877431] [image_resize_right]: [NitrosNode]
Calling user's pre-load-graph callback
[component_container-2] [INFO] [1701322796.648893048] [image_resize_right]: [NitrosNode]
Loading application
[component_container-2] [INFO] [1701322796.648910040] [image_resize_right]:
[NitrosContext] Loading application:
'/tmp/isaac_ros_nitros/graphs/JTILYGRDJC/JTILYGRDJC.yaml'
[component_container-2] [INFO] [1701322796.655138892] [image_resize_right]: [NitrosNode]
Linking Nitros pub/sub to the loaded application
[component_container-2] [INFO] [1701322796.656533845] [image_resize_right]: [NitrosNode]
Calling user's post-load-graph callback
[component_container-2] [INFO] [1701322796.656981858] [image_resize_right]: [ResizeNode]
postLoadGraphCallback().
[component_container-2] [INFO] [1701322796.657452719] [image_resize_right]:
[NitrosContext] Initializing application...
[component_container-2] [INFO] [1701322797.361305149] [image_resize_right]:
[NitrosContext] Running application...
[component_container-2] [INFO] [1701322797.361897198] [image_resize_right]: [NitrosNode]
Starting a heartbeat timer (eid=17)
[realsense2_camera_node-1] [WARN] [1701322800.764950763] [camera.camera]: re-enable
the stream for the change to take effect.
[realsense2_camera_node-1] [WARN] [1701322802.145945524] [camera.camera]: re-enable
the stream for the change to take effect.
[realsense2_camera_node-1] [INFO] [1701322802.212011146] [camera.camera]: Stopping
Sensor: Stereo Module
[realsense2_camera_node-1] [INFO] [1701322802.422906682] [camera.camera]: Starting
Sensor: Stereo Module
[realsense2_camera_node-1] [INFO] [1701322802.686489277] [camera.camera]: Open profile:
stream_type: Depth(0), Format: Z16, Width: 640, Height: 480, FPS: 90
[realsense2_camera_node-1] [INFO] [1701322802.686779301] [camera.camera]: Stopping
Sensor: RGB Camera
[realsense2_camera_node-1] [INFO] [1701322802.715066711] [camera.camera]: Starting
Sensor: RGB Camera
[realsense2_camera_node-1] [WARN] [1701322802.740877985] [camera.camera]:
[realsense2_camera_node-1] [INFO] [1701322802.747071828] [camera.camera]: Open profile:

stream_type: Color(0), Format: RGB8, Width: 1280, Height: 720, FPS: 30
[realsense2_camera_node-1] [INFO] [1701322802.748940746] [camera.camera]: RealSense
Node Is Up!

Command

ros2 launch isaac_ros_yolov8 isaac_ros_yolov8_visualize.launch.py model_file_path:=/workspaces/isaac_ros-dev/src/onnx/drone_detection_v21_int8.onnx engine_file_path:=/workspaces/isaac_ros-dev/src/onnx/drone_detection_v21_int8.plan input_binding_names:=['images'] output_binding_names:=['output0'] network_image_width:=640 network_image_height:=640 force_engine_update:=False image_mean:=[0.0,0.0,0.0] image_stddev:=[1.0,1.0,1.0] input_image_width:=640 input_image_height:=640 confidence_threshold:=0.25 nms_threshold:=0.45

Output

[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-11-30-08-39-38-
677274-ubuntu-563002
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container_mt-1]: process started with pid [563013]
[INFO] [isaac_ros_yolov8_visualizer.py-2]: process started with pid [563015]
[component_container_mt-1] [INFO] [1701322779.374554982] [tensor_rt_container]: Load
Library: /workspaces/isaac_ros-
dev/install/isaac_ros_dnn_image_encoder/lib/libdnn_image_encoder_node.so
[component_container_mt-1] [INFO] [1701322779.422003422] [tensor_rt_container]: Found
class:
rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::DnnImageEncod
erNode>
[component_container_mt-1] [INFO] [1701322779.422146370] [tensor_rt_container]:
Instantiate class:
rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::DnnImageEncod
erNode>
[component_container_mt-1] [INFO] [1701322779.428988104] [NitrosContext]:
[NitrosContext] Creating a new shared context
[component_container_mt-1] [INFO] [1701322779.429260784] [dnn_image_encoder]:
[NitrosNode] Initializing NitrosNode
[component_container_mt-1] [INFO] [1701322779.430896223] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so
[component_container_mt-1] [INFO] [1701322779.439542456] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/libgxf_gxf_helpers.so
[component_container_mt-1] [INFO] [1701322779.446296123] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/libgxf_sight.so
[component_container_mt-1] [INFO] [1701322779.451424463] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/libgxf_atlas.so
[component_container_mt-1] [INFO] [1701322779.456436031] [NitrosContext]:
[NitrosContext] Loading application: '/workspaces/isaac_ros-
dev/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_grap
h.yaml'
[component_container_mt-1] [INFO] [1701322779.457807943] [NitrosContext]:
[NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1701322779.462127075] [NitrosContext]:
[NitrosContext] Running application...

[component_container_mt-1] 2023-11-30 08:39:39.462 WARN gxf/std/program.cpp@514: No
system specified. Nothing to do
[component_container_mt-1] [INFO] [1701322779.463995417] [dnn_image_encoder]:
[NitrosNode] Starting NitrosNode
[component_container_mt-1] [INFO] [1701322779.464062075] [dnn_image_encoder]:
[NitrosNode] Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1701322779.467213334] [dnn_image_encoder]:
[NitrosNode] Loading built-in extension specs
[component_container_mt-1] [INFO] [1701322779.467351162] [dnn_image_encoder]:
[NitrosNode] Loading preset extension specs
[component_container_mt-1] [INFO] [1701322779.469741855] [dnn_image_encoder]:
[NitrosNode] Loading extension specs
[component_container_mt-1] [INFO] [1701322779.469813601] [dnn_image_encoder]:
[NitrosNode] Loading generator rules
[component_container_mt-1] [INFO] [1701322779.470381681] [dnn_image_encoder]:
[NitrosNode] Loading extensions
[component_container_mt-1] [INFO] [1701322779.471138759] [dnn_image_encoder]:
[NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so
[component_container_mt-1] [INFO] [1701322779.473786516] [dnn_image_encoder]:
[NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so
[component_container_mt-1] [INFO] [1701322779.475630569] [dnn_image_encoder]:
[NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so
[component_container_mt-1] [INFO] [1701322779.482131012] [dnn_image_encoder]:
[NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so
[component_container_mt-1] [INFO] [1701322779.496469250] [NitrosContext]:
[NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so
[component_container_mt-1] [INFO] [1701322779.498295478] [dnn_image_encoder]:
[NitrosNode] Loading graph to the optimizer
[component_container_mt-1] [INFO] [1701322779.506178682] [dnn_image_encoder]:
[NitrosNode] Running optimization
[component_container_mt-1] [INFO] [1701322779.755889691] [dnn_image_encoder]:
[NitrosNode] Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1701322779.765680566] [dnn_image_encoder]:
[NitrosNode] Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1701322779.793095948] [dnn_image_encoder]:
[NitrosNode] Starting negotiation...
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/dnn_image_encoder' in
container '/tensor_rt_container'
[component_container_mt-1] [INFO] [1701322779.800116631] [tensor_rt_container]: Load
Library: /workspaces/isaac_ros-dev/install/isaac_ros_tensor_rt/lib/libtensor_rt_node.so
[component_container_mt-1] [INFO] [1701322779.854147021] [tensor_rt_container]: Found
class:
rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TensorRTNode>
[component_container_mt-1] [INFO] [1701322779.854299506] [tensor_rt_container]:

Instantiate class:
rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::dnn_inference::TensorRTNode>
[component_container_mt-1] [INFO] [1701322779.870576999] [tensor_rt]: [NitrosNode]
Initializing NitrosNode
[component_container_mt-1] [INFO] [1701322779.873076367] [tensor_rt]: [NitrosNode]
Starting NitrosNode
[component_container_mt-1] [INFO] [1701322779.873117968] [tensor_rt]: [NitrosNode]
Loading built-in preset extension specs
[component_container_mt-1] [INFO] [1701322779.876811227] [tensor_rt]: [NitrosNode]
Loading built-in extension specs
[component_container_mt-1] [INFO] [1701322779.876909438] [tensor_rt]: [NitrosNode]
Loading preset extension specs
[component_container_mt-1] [INFO] [1701322779.878036990] [tensor_rt]: [NitrosNode]
Loading extension specs
[component_container_mt-1] [INFO] [1701322779.878084640] [tensor_rt]: [NitrosNode]
Loading generator rules
[component_container_mt-1] [INFO] [1701322779.878297030] [tensor_rt]: [NitrosNode]
Loading extensions
[component_container_mt-1] [INFO] [1701322779.882214999] [tensor_rt]: [NitrosContext]
Loading extension: gxf/tensor_rt/libgxf_tensor_rt.so
[component_container_mt-1] [INFO] [1701322779.886617110] [tensor_rt]: [NitrosNode]
Loading graph to the optimizer
[component_container_mt-1] [INFO] [1701322779.890131195] [tensor_rt]: [NitrosNode]
Running optimization
[component_container_mt-1] [INFO] [1701322779.920149533] [tensor_rt]: [NitrosNode]
Obtaining graph IO group info from the optimizer
[component_container_mt-1] [INFO] [1701322779.927303211] [tensor_rt]: [NitrosNode]
Creating negotiated publishers/subscribers
[component_container_mt-1] [INFO] [1701322779.928922938] [tensor_rt]:
[NitrosPublisherSubscriberGroup] Pinning the component "inference/rx"
(type="nvidia::gxf::DoubleBufferReceiver") to use its compatible format only:
"nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1701322779.929025181] [tensor_rt]:
[NitrosPublisherSubscriberGroup] Pinning the component "sink/sink"
(type="nvidia::isaac_ros::MessageRelay") to use its compatible format only:
"nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1701322779.937657302] [tensor_rt]: [NitrosNode]
Starting negotiation...
[component_container_mt-1] [INFO] [1701322779.939187458] [dnn_image_encoder]:
Negotiating
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/tensor_rt' in container
'/tensor_rt_container'
[component_container_mt-1] [INFO] [1701322779.943489278] [tensor_rt_container]: Load
Library: /workspaces/isaac_ros-dev/install/isaac_ros_yolov8/lib/libyolov8_decoder_node.so

[component_container_mt-1] [INFO] [1701322779.967102151] [tensor_rt_container]: Found
class:
rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::yolov8::YoloV8DecoderNode>
[component_container_mt-1] [INFO] [1701322779.968357835] [tensor_rt_container]:
Instantiate class:
rclcpp_components::NodeFactoryTemplate<nvidia::isaac_ros::yolov8::YoloV8DecoderNode>
[component_container_mt-1] [INFO] [1701322779.991981876]
[yolov8_decoder_node.ManagedNitrosSubscriber]: Starting Managed Nitros Subscriber
[component_container_mt-1] [INFO] [1701322779.992318590] [tensor_rt]: Negotiating
[component_container_mt-1] [INFO] [1701322779.993687781] [tensor_rt]: Could not
negotiate
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/yolov8_decoder_node' in
container '/tensor_rt_container'
[component_container_mt-1] [INFO] [1701322780.795652960] [dnn_image_encoder]:
[NitrosNode] Starting post negotiation setup
[component_container_mt-1] [INFO] [1701322780.795782660] [dnn_image_encoder]:
[NitrosNode] Getting data format negotiation results
[component_container_mt-1] [INFO] [1701322780.795815109] [dnn_image_encoder]:
[NitrosPublisher] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1701322780.795831846] [dnn_image_encoder]:
[NitrosSubscriber] Negotiation ended with no results
[component_container_mt-1] [INFO] [1701322780.795843174] [dnn_image_encoder]:
[NitrosSubscriber] Use the compatible subscriber: topic_name="/image",
data_format="nitros_image_bgr8"
[component_container_mt-1] [INFO] [1701322780.796284371] [dnn_image_encoder]:
[NitrosNode] Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1701322780.912005116] [dnn_image_encoder]:
[NitrosNode] Wrote the final top level YAML graph to
"/tmp/isaac_ros_nitros/graphs/JNFIVUAIDJ/JNFIVUAIDJ.yaml"
[component_container_mt-1] [INFO] [1701322780.914359872] [dnn_image_encoder]:
[NitrosNode] Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1701322780.915398238] [dnn_image_encoder]: In DNN
Image Encoder Node preLoadGraphCallback().
[component_container_mt-1] [INFO] [1701322780.916077970] [dnn_image_encoder]:
[NitrosNode] Loading application
[component_container_mt-1] [INFO] [1701322780.916602529] [dnn_image_encoder]:
[NitrosContext] Loading application:
'/tmp/isaac_ros_nitros/graphs/JNFIVUAIDJ/JNFIVUAIDJ.yaml'
[component_container_mt-1] [INFO] [1701322780.926448733] [dnn_image_encoder]:
[NitrosNode] Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1701322780.928209296] [dnn_image_encoder]:
[NitrosNode] Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1701322780.928670525] [dnn_image_encoder]: In DNN
Image Encoder Node postLoadGraphCallback().

[component_container_mt-1] [INFO] [1701322780.929245582] [dnn_image_encoder]:
[NitrosContext] Initializing application...
[component_container_mt-1] [INFO] [1701322780.939570583] [tensor_rt]: [NitrosNode]
Starting post negotiation setup
[component_container_mt-1] [INFO] [1701322780.939669530] [tensor_rt]: [NitrosNode]
Getting data format negotiation results
[component_container_mt-1] [INFO] [1701322780.939688123] [tensor_rt]: [NitrosPublisher]
Negotiation ended with no results
[component_container_mt-1] [INFO] [1701322780.939704251] [tensor_rt]: [NitrosPublisher]
Use only the compatible publisher: topic_name="/tensor_sub",
data_format="nitros_tensor_list_nhwc_rgb_f32"
[component_container_mt-1] [INFO] [1701322780.939790558] [tensor_rt]: [NitrosSubscriber]
Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32"
[component_container_mt-1] [INFO] [1701322780.940029957] [tensor_rt]: [NitrosNode]
Exporting the final graph based on the negotiation results
[component_container_mt-1] [INFO] [1701322780.944896561] [tensor_rt]: [NitrosNode]
Wrote the final top level YAML graph to
"/tmp/isaac_ros_nitros/graphs/IQPYNYPNPH/IQPYNYPNPH.yaml"
[component_container_mt-1] [INFO] [1701322780.945559780] [tensor_rt]: [NitrosNode]
Calling user's pre-load-graph callback
[component_container_mt-1] [INFO] [1701322780.945582277] [tensor_rt]: [NitrosNode]
Loading application
[component_container_mt-1] [INFO] [1701322781.087987504] [dnn_image_encoder]:
[NitrosContext] Running application...
[component_container_mt-1] [INFO] [1701322781.089617119] [tensor_rt]: [NitrosContext]
Loading application: '/tmp/isaac_ros_nitros/graphs/IQPYNYPNPH/IQPYNYPNPH.yaml'
[component_container_mt-1] [INFO] [1701322781.093182182] [tensor_rt]: [NitrosNode]
Linking Nitros pub/sub to the loaded application
[component_container_mt-1] [INFO] [1701322781.093334283] [tensor_rt]: [NitrosNode]
Calling user's post-load-graph callback
[component_container_mt-1] [INFO] [1701322781.093362635] [tensor_rt]: In TensorRTNode
postLoadGraphCallback().
[component_container_mt-1] 2023-11-30 08:39:41.092 WARN
gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'dummy_rx' in component
''.
[component_container_mt-1] 2023-11-30 08:39:41.093 WARN
gxf/std/yaml_file_loader.cpp@1058: Using unregistered parameter 'dev_id' in component
'stream'.
[component_container_mt-1] [INFO] [1701322781.102424465] [dnn_image_encoder]:
[NitrosNode] Starting a heartbeat timer (eid=25)
[component_container_mt-1] [INFO] [1701322784.080123808] [tensor_rt]: Read tensor shape
information from TRT Model Engine: /workspaces/isaac_ros-
dev/src/onnx/drone_detection_v22_Yolov8n_int8.plan
[component_container_mt-1] [INFO] [1701322784.081762767] [tensor_rt]: Tensors 201600

bytes, num outputs 40 x tensors per output 1 = 40 blocks
[component_container_mt-1] [INFO] [1701322784.082625256] [tensor_rt]: [NitrosContext]
Initializing application...
[component_container_mt-1] [INFO] [1701322784.088133479] [tensor_rt]: [NitrosContext]
Running application...
[component_container_mt-1] [INFO] [1701322784.090031262] [tensor_rt]: [NitrosNode]
Starting a heartbeat timer (eid=110)
[component_container_mt-1] [INFO] [1701322795.652244513] [dnn_image_encoder]:
Negotiating

list of nodes

/camera/camera
/dnn_image_encoder
/image_resize_right
/launch_ros_563002
/launch_ros_563055
/resize_launch_container
/tensor_rt
/tensor_rt_container
/yolov8_decoder_node
/yolov8_visualizer

list of topics

/camera/aligned_depth_to_color/camera_info
/camera/aligned_depth_to_color/image_raw
/camera/aligned_depth_to_color/image_raw/compressed
/camera/aligned_depth_to_color/image_raw/compressedDepth
/camera/aligned_depth_to_color/image_raw/theora
/camera/color/camera_info
/camera/color/camera_info/nitros
/camera/color/camera_info_resize
/camera/color/camera_info_resize/nitros
/camera/color/image_raw
/camera/color/image_raw/compressed
/camera/color/image_raw/compressedDepth
/camera/color/image_raw/nitros
/camera/color/image_raw/theora

/camera/color/metadata
/camera/depth/camera_info
/camera/depth/image_rect_raw
/camera/depth/image_rect_raw/compressed
/camera/depth/image_rect_raw/compressedDepth
/camera/depth/image_rect_raw/theora
/camera/depth/metadata
/camera/extrinsics/depth_to_color
/camera/extrinsics/depth_to_depth
/camera/imu
/detections_output
/image
/image/nitros
/image/nitros/nitros_image_rgb8
/parameter_events
/rosout
/tensor_pub
/tensor_pub/nitros
/tensor_sub
/tensor_sub/nitros
/tf_static
/yolov8_processed_image

I hope I can get some help on how to solve this as there would be more applications running alongside the yolov8.

Thanks.

mzahana avatar Nov 30 '23 06:11 mzahana

In the TensorRTNode and the DNNImageEncoderNode, there's a parameter called num_blocks. This defaults to 40. Could you try reducing it to say 20 or 10 and seeing if that improves the situation?

kajananchinniahNV avatar Nov 30 '23 19:11 kajananchinniahNV

@kajananchinniahNV I tried num_blocks 20 and 10, but the issue is still there. I need to run both isaac_ros_visual_slam and yolov8. If I run visual slam alone, I don't get the over-current issue. If I run yolov8 alone, I get the over-current issue. If I run both, I get the over-current issue.

mzahana avatar Dec 01 '23 17:12 mzahana

Hello,

I am working with @mzahana in the same project, and I added the ros bag for running both isaac_ros_vislam and isaac_ros_yolov8 and you can see the poor performance for the Vislam and the discontinuity. Also, added a ros bag for just running isaac_ros_yolov8 and you can see the lag in the detection.

ros bag for vislam and yolov8 Here

ros bag for running yolov8 only Here

To clarify, we use our own model that we trained on Yolov8n.

Also, the isaac_ros_yolov8 package accept only input image with 640x640 size. So, in the isaac_ros_vislam launch file, we added a node that to resize the image to 640x640

you can see the launch file in the first:

# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
# ... (your license text)

import launch
from launch_ros.actions import ComposableNodeContainer, Node
from launch_ros.descriptions import ComposableNode

def generate_launch_description():
    """Launch file which brings up visual slam node configured for RealSense."""
    realsense_camera_node = Node(
        name='camera',
        namespace='camera',
        package='realsense2_camera',
        executable='realsense2_camera_node',
        parameters=[{
                'enable_infra1': True,
                'enable_infra2': True,
                'enable_color': True,
                'enable_depth': True,
                'depth_module.emitter_enabled': 0,
                'depth_module.profile': '640x360x90',
                'enable_gyro': True,
                'enable_accel': True,
                'gyro_fps': 200,
                'accel_fps': 200,
                'unite_imu_method': 2
        }]
    )

    visual_slam_node = ComposableNode(
        name='visual_slam_node',
        package='isaac_ros_visual_slam',
        plugin='nvidia::isaac_ros::visual_slam::VisualSlamNode',
        parameters=[{
                    'denoise_input_images': False,
                    'rectified_images': True,
                    'enable_debug_mode': False,
                    'debug_dump_path': '/tmp/cuvslam',
                    'enable_slam_visualization': True,
                    'enable_landmarks_view': True,
                    'enable_observations_view': True,
                    'map_frame': 'map',
                    'odom_frame': 'odom',
                    'base_frame': 'camera_link',
                    'input_imu_frame': 'camera_gyro_optical_frame',
                    'enable_imu_fusion': True,
                    'gyro_noise_density': 0.000244,
                    'gyro_random_walk': 0.000019393,
                    'accel_noise_density': 0.001862,
                    'accel_random_walk': 0.003,
                    'calibration_frequency': 200.0,
                    'img_jitter_threshold_ms': 22.00
                    }],
        remappings=[('stereo_camera/left/image', 'camera/infra1/image_rect_raw'),
                    ('stereo_camera/left/camera_info', 'camera/infra1/camera_info'),
                    ('stereo_camera/right/image', 'camera/infra2/image_rect_raw'),
                    ('stereo_camera/right/camera_info', 'camera/infra2/camera_info'),
                    ('visual_slam/imu', 'camera/imu')]
    )

    visual_slam_launch_container = ComposableNodeContainer(
        name='visual_slam_launch_container',
        namespace='',
        package='rclcpp_components',
        executable='component_container',
        composable_node_descriptions=[
            visual_slam_node
        ]
    )
       
    image_resize_node_color = ComposableNode(
        package='isaac_ros_image_proc',
        plugin='nvidia::isaac_ros::image_proc::ResizeNode',
        name='image_resize_right',
        parameters=[{
                'output_width': 640,
                'output_height': 640,
                'keep_aspect_ratio': True
        }],
        remappings=[
            ('camera_info', '/camera/color/camera_info'),
            ('image', '/camera/color/image_raw'),
            ('resize/camera_info', '/camera/color/camera_info_resize'),
            ('resize/image', '/image')]
    )
    
    resize_launch_container = ComposableNodeContainer(
        name='resize_launch_container',
        namespace='',
        package='rclcpp_components',
        executable='component_container',
        composable_node_descriptions=[
            image_resize_node_color
        ],
        output='screen'
    )

    return launch.LaunchDescription([visual_slam_launch_container, realsense_camera_node, resize_launch_container])

Thank you.

AbdullahGM1 avatar Dec 04 '23 10:12 AbdullahGM1

Please take a look at these forum posts regarding this issue

  1. https://forums.developer.nvidia.com/t/system-throttled-due-to-over-current-on-orin-nx/247300
  2. Search results with similar problem

swapnesh-wani-nvidia avatar Dec 04 '23 23:12 swapnesh-wani-nvidia

I solved the issue,

I followed the procedures in this post

AbdullahGM1 avatar Dec 11 '23 06:12 AbdullahGM1