isaac_ros_image_segmentation icon indicating copy to clipboard operation
isaac_ros_image_segmentation copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA-ISAAC-ROS

how to fix the input and output binding for a custom trt engine ,

Open BADAL244 opened this issue 1 year ago • 12 comments

ros2 launch isaac_ros_unet isaac_ros_unet_tensor_rt.launch.py model_name:=peoplesemsegnet_shuffleseg engine_file_path:=/home/badal/data/new_engine.plan input_binding_names:=['input_2:0'] output_binding_names:=['argmax_1'] network_output_type:='argmax' rvi[INFO] [launch]: All log files can be found below /home/badal/.ros/log/2023-07-25-18-56-14-386452-badal-desktop-4707 [INFO] [launch]: Default logging verbosity is set to INFO [INFO] [component_container_mt-1]: process started with pid [4770] [component_container_mt-1] [INFO] [1690282576.766031488] [unet_container]: Load Library: /home/badal/vslam_ws/install/isaac_ros_dnn_encoders/lib/libdnn_image_encoder_node.so [component_container_mt-1] [INFO] [1690282577.418500576] [unet_container]: Found class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::dnn_inference::DnnImageEncoderNode [component_container_mt-1] [INFO] [1690282577.418960256] [unet_container]: Instantiate class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::dnn_inference::DnnImageEncoderNode [component_container_mt-1] [INFO] [1690282577.474196192] [NitrosContext]: [NitrosContext] Creating a new shared context [component_container_mt-1] [INFO] [1690282577.475289536] [dnn_image_encoder]: [NitrosNode] Initializing NitrosNode [component_container_mt-1] [INFO] [1690282577.499201984] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/std/libgxf_std.so [component_container_mt-1] [INFO] [1690282577.536095968] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_gxf_helpers.so [component_container_mt-1] [INFO] [1690282577.564563936] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_sight.so [component_container_mt-1] [INFO] [1690282577.594015360] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/libgxf_atlas.so [component_container_mt-1] [INFO] [1690282577.624380416] [NitrosContext]: [NitrosContext] Loading application: '/home/badal/vslam_ws/install/isaac_ros_nitros/share/isaac_ros_nitros/config/type_adapter_nitros_context_graph.yaml' [component_container_mt-1] [INFO] [1690282577.633416000] [NitrosContext]: [NitrosContext] Initializing application... [component_container_mt-1] [INFO] [1690282577.648229568] [NitrosContext]: [NitrosContext] Running application... [component_container_mt-1] 2023-07-25 18:56:17.648 WARN gxf/std/program.cpp@456: No system specified. Nothing to do [component_container_mt-1] [INFO] [1690282577.673352704] [dnn_image_encoder]: [NitrosNode] Starting NitrosNode [component_container_mt-1] [INFO] [1690282577.673727968] [dnn_image_encoder]: [NitrosNode] Loading built-in preset extension specs [component_container_mt-1] [INFO] [1690282577.695671456] [dnn_image_encoder]: [NitrosNode] Loading built-in extension specs [component_container_mt-1] [INFO] [1690282577.696311520] [dnn_image_encoder]: [NitrosNode] Loading preset extension specs [component_container_mt-1] [INFO] [1690282577.712717824] [dnn_image_encoder]: [NitrosNode] Loading extension specs [component_container_mt-1] [INFO] [1690282577.713155712] [dnn_image_encoder]: [NitrosNode] Loading generator rules [component_container_mt-1] [INFO] [1690282577.716584992] [dnn_image_encoder]: [NitrosNode] Loading extensions [component_container_mt-1] [INFO] [1690282577.726539072] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/libgxf_message_compositor.so [component_container_mt-1] [INFO] [1690282577.750451488] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/cuda/libgxf_cuda.so [component_container_mt-1] [INFO] [1690282577.778020864] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/serialization/libgxf_serialization.so [component_container_mt-1] [INFO] [1690282577.811581536] [dnn_image_encoder]: [NitrosContext] Loading extension: gxf/lib/image_proc/libgxf_tensorops.so [component_container_mt-1] [INFO] [1690282577.928038944] [NitrosContext]: [NitrosContext] Loading extension: gxf/lib/multimedia/libgxf_multimedia.so [component_container_mt-1] [INFO] [1690282577.941050720] [dnn_image_encoder]: [NitrosNode] Loading graph to the optimizer [component_container_mt-1] [INFO] [1690282577.998541280] [dnn_image_encoder]: [NitrosNode] Running optimization [component_container_mt-1] [INFO] [1690282578.889717216] [dnn_image_encoder]: [NitrosNode] Obtaining graph IO group info from the optimizer [component_container_mt-1] [INFO] [1690282578.918500544] [dnn_image_encoder]: [NitrosNode] Creating negotiated publishers/subscribers [component_container_mt-1] [INFO] [1690282579.034744352] [dnn_image_encoder]: [NitrosNode] Starting negotiation... [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/dnn_image_encoder' in container '/unet_container' [component_container_mt-1] [INFO] [1690282579.073490656] [unet_container]: Load Library: /home/badal/vslam_ws/install/isaac_ros_tensor_rt/lib/libtensor_rt_node.so [component_container_mt-1] [INFO] [1690282579.087640320] [unet_container]: Found class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::dnn_inference::TensorRTNode [component_container_mt-1] [INFO] [1690282579.088165504] [unet_container]: Instantiate class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::dnn_inference::TensorRTNode [component_container_mt-1] [INFO] [1690282579.166642240] [tensor_rt_node]: [NitrosNode] Initializing NitrosNode [component_container_mt-1] [INFO] [1690282579.208608416] [tensor_rt_node]: [TensorRTNode] Set input data format to: "nitros_tensor_list_nchw_rgb_f32" [component_container_mt-1] [INFO] [1690282579.208914752] [tensor_rt_node]: [TensorRTNode] Set output data format to: "nitros_tensor_list_nhwc_rgb_f32" [component_container_mt-1] [INFO] [1690282579.209256000] [tensor_rt_node]: [NitrosNode] Starting NitrosNode [component_container_mt-1] [INFO] [1690282579.209385184] [tensor_rt_node]: [NitrosNode] Loading built-in preset extension specs [component_container_mt-1] [INFO] [1690282579.226240448] [tensor_rt_node]: [NitrosNode] Loading built-in extension specs [component_container_mt-1] [INFO] [1690282579.226681248] [tensor_rt_node]: [NitrosNode] Loading preset extension specs [component_container_mt-1] [INFO] [1690282579.229786432] [tensor_rt_node]: [NitrosNode] Loading extension specs [component_container_mt-1] [INFO] [1690282579.230094560] [tensor_rt_node]: [NitrosNode] Loading generator rules [component_container_mt-1] [INFO] [1690282579.232664544] [tensor_rt_node]: [NitrosNode] Loading extensions [component_container_mt-1] [INFO] [1690282579.275218304] [tensor_rt_node]: [NitrosContext] Loading extension: gxf/tensor_rt/libgxf_tensor_rt.so [component_container_mt-1] [INFO] [1690282579.584771456] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set. [component_container_mt-1] [INFO] [1690282579.596685632] [tensor_rt_node]: [NitrosNode] Loading graph to the optimizer [component_container_mt-1] [INFO] [1690282579.600179584] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set. [component_container_mt-1] [INFO] [1690282579.610728960] [tensor_rt_node]: [NitrosNode] Running optimization [component_container_mt-1] [INFO] [1690282579.621694016] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set. [component_container_mt-1] [INFO] [1690282579.675447456] [tensor_rt_node]: [NitrosNode] Obtaining graph IO group info from the optimizer [component_container_mt-1] [INFO] [1690282579.679445600] [tensor_rt_node]: [NitrosNode] Creating negotiated publishers/subscribers [component_container_mt-1] [INFO] [1690282579.679762656] [tensor_rt_node]: [NitrosPublisherSubscriberGroup] Pinning the component "inference/rx" (type="nvidia::gxf::DoubleBufferReceiver") to use its compatible format only: "nitros_tensor_list_nchw_rgb_f32" [component_container_mt-1] [INFO] [1690282579.680097824] [tensor_rt_node]: [NitrosPublisherSubscriberGroup] Pinning the component "sink/sink" (type="nvidia::isaac_ros::MessageRelay") to use its compatible format only: "nitros_tensor_list_nhwc_rgb_f32" [component_container_mt-1] [INFO] [1690282579.694474464] [tensor_rt_node]: [NitrosNode] Starting negotiation... [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/tensor_rt_node' in container '/unet_container' [component_container_mt-1] [INFO] [1690282579.699725760] [dnn_image_encoder]: Negotiating [component_container_mt-1] [INFO] [1690282579.717791808] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set. [component_container_mt-1] [INFO] [1690282579.722061536] [unet_container]: Load Library: /home/badal/vslam_ws/install/isaac_ros_unet/lib/libunet_decoder_node.so [component_container_mt-1] [INFO] [1690282579.742071616] [unet_container]: Found class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::unet::UNetDecoderNode [component_container_mt-1] [INFO] [1690282579.742371008] [unet_container]: Instantiate class: rclcpp_components::NodeFactoryTemplatenvidia::isaac_ros::unet::UNetDecoderNode [component_container_mt-1] [INFO] [1690282579.782681024] [unet_decoder_node]: [NitrosNode] Initializing NitrosNode [component_container_mt-1] [INFO] [1690282579.804838176] [unet_decoder_node]: [NitrosNode] Starting NitrosNode [component_container_mt-1] [INFO] [1690282579.805103424] [unet_decoder_node]: [NitrosNode] Loading built-in preset extension specs [component_container_mt-1] [INFO] [1690282579.815235968] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set. [component_container_mt-1] [INFO] [1690282579.819626336] [unet_decoder_node]: [NitrosNode] Loading built-in extension specs [component_container_mt-1] [INFO] [1690282579.820016384] [unet_decoder_node]: [NitrosNode] Loading preset extension specs [component_container_mt-1] [INFO] [1690282579.824074848] [unet_decoder_node]: [NitrosNode] Loading extension specs [component_container_mt-1] [INFO] [1690282579.824297824] [unet_decoder_node]: [NitrosNode] Loading generator rules [component_container_mt-1] [INFO] [1690282579.824387040] [unet_decoder_node]: [NitrosNode] Loading extensions [component_container_mt-1] [INFO] [1690282579.854912736] [unet_decoder_node]: [NitrosContext] Loading extension: gxf/libgxf_segmentation_postprocessor.so [component_container_mt-1] [INFO] [1690282579.878305888] [unet_decoder_node]: [NitrosNode] Loading graph to the optimizer [component_container_mt-1] [INFO] [1690282579.897539936] [unet_decoder_node]: [NitrosNode] Running optimization [component_container_mt-1] [INFO] [1690282579.922508416] [dnn_image_encoder]: [NitrosSubscriber] Received a message but the application receiver's pointer is not yet set. [component_container_mt-1] [INFO] [1690282580.043207552] [dnn_image_encoder]: [NitrosNode] Starting post negotiation setup [component_container_mt-1] [INFO] [1690282580.044096832] [dnn_image_encoder]: [NitrosNode] Getting data format negotiation results [component_container_mt-1] [INFO] [1690282580.044571904] [dnn_image_encoder]: [NitrosPublisher] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32" [component_container_mt-1] [INFO] [1690282580.044825984] [dnn_image_encoder]: [NitrosSubscriber] Negotiation ended with no results [component_container_mt-1] [INFO] [1690282580.045043456] [dnn_image_encoder]: [NitrosSubscriber] Use the compatible subscriber: topic_name="/image", data_format="nitros_image_bgr8" [component_container_mt-1] [INFO] [1690282580.045369952] [dnn_image_encoder]: [NitrosNode] Exporting the final graph based on the negotiation results [component_container_mt-1] [INFO] [1690282580.105343104] [dnn_image_encoder]: [NitrosNode] Wrote the final top level YAML graph to "/home/badal/vslam_ws/install/isaac_ros_dnn_encoders/share/isaac_ros_dnn_encoders/YXBOCKVNMB.yaml" [component_container_mt-1] [INFO] [1690282580.106741440] [dnn_image_encoder]: [NitrosNode] Calling user's pre-load-graph callback [component_container_mt-1] [INFO] [1690282580.106968288] [dnn_image_encoder]: In DNN Image Encoder Node preLoadGraphCallback(). [component_container_mt-1] [INFO] [1690282580.107284608] [dnn_image_encoder]: [NitrosNode] Loading application [component_container_mt-1] [INFO] [1690282580.107480672] [dnn_image_encoder]: [NitrosContext] Loading application: '/home/badal/vslam_ws/install/isaac_ros_dnn_encoders/share/isaac_ros_dnn_encoders/YXBOCKVNMB.yaml' [component_container_mt-1] [INFO] [1690282580.135827552] [dnn_image_encoder]: [NitrosNode] Linking Nitros pub/sub to the loaded application [component_container_mt-1] [INFO] [1690282580.137354688] [unet_decoder_node]: [NitrosNode] Obtaining graph IO group info from the optimizer [component_container_mt-1] [INFO] [1690282580.137629664] [dnn_image_encoder]: [NitrosNode] Calling user's post-load-graph callback [component_container_mt-1] [INFO] [1690282580.137811776] [dnn_image_encoder]: In DNN Image Encoder Node postLoadGraphCallback(). [component_container_mt-1] [INFO] [1690282580.138377088] [dnn_image_encoder]: [NitrosContext] Initializing application... [component_container_mt-1] [INFO] [1690282580.152343392] [unet_decoder_node]: [NitrosNode] Creating negotiated publishers/subscribers [component_container_mt-1] [INFO] [1690282580.153414752] [unet_decoder_node]: [NitrosPublisherSubscriberGroup] Pinning the component "colored_segmentation_mask_sink/sink" (type="nvidia::isaac_ros::MessageRelay") to use its compatible format only: "nitros_image_rgb8" [component_container_mt-1] [INFO] [1690282580.181691168] [unet_decoder_node]: [NitrosNode] Starting negotiation... [INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/unet_decoder_node' in container '/unet_container' [component_container_mt-1] [INFO] [1690282580.187514016] [tensor_rt_node]: Negotiating [component_container_mt-1] [INFO] [1690282580.202962944] [dnn_image_encoder]: [NitrosContext] Running application... [component_container_mt-1] [INFO] [1690282580.203566240] [dnn_image_encoder]: [NitrosNode] Starting a heartbeat timer (eid=46) [component_container_mt-1] [INFO] [1690282580.219748832] [dnn_image_encoder]: Negotiating [component_container_mt-1] [INFO] [1690282580.697649376] [tensor_rt_node]: [NitrosNode] Starting post negotiation setup [component_container_mt-1] [INFO] [1690282580.697998720] [tensor_rt_node]: [NitrosNode] Getting data format negotiation results [component_container_mt-1] [INFO] [1690282580.698143520] [tensor_rt_node]: [NitrosPublisher] Use the negotiated data format: "nitros_tensor_list_nhwc_rgb_f32" [component_container_mt-1] [INFO] [1690282580.698274336] [tensor_rt_node]: [NitrosSubscriber] Use the negotiated data format: "nitros_tensor_list_nchw_rgb_f32" [component_container_mt-1] [INFO] [1690282580.698429440] [tensor_rt_node]: [NitrosNode] Exporting the final graph based on the negotiation results [component_container_mt-1] [INFO] [1690282580.717308256] [tensor_rt_node]: [NitrosNode] Wrote the final top level YAML graph to "/home/badal/vslam_ws/install/isaac_ros_tensor_rt/share/isaac_ros_tensor_rt/JROAGQYGZR.yaml" [component_container_mt-1] [INFO] [1690282580.717614592] [tensor_rt_node]: [NitrosNode] Calling user's pre-load-graph callback [component_container_mt-1] [INFO] [1690282580.717712448] [tensor_rt_node]: [NitrosNode] Loading application [component_container_mt-1] [INFO] [1690282580.717825056] [tensor_rt_node]: [NitrosContext] Loading application: '/home/badal/vslam_ws/install/isaac_ros_tensor_rt/share/isaac_ros_tensor_rt/JROAGQYGZR.yaml' [component_container_mt-1] 2023-07-25 18:56:20.727 WARN gxf/std/yaml_file_loader.cpp@952: Using unregistered parameter 'dummy_rx' in component ''. [component_container_mt-1] [INFO] [1690282580.730695520] [tensor_rt_node]: [NitrosNode] Linking Nitros pub/sub to the loaded application [component_container_mt-1] [INFO] [1690282580.731416160] [tensor_rt_node]: [NitrosNode] Calling user's post-load-graph callback [component_container_mt-1] [INFO] [1690282580.731563776] [tensor_rt_node]: In TensorRTNode postLoadGraphCallback(). [component_container_mt-1] [INFO] [1690282580.732454208] [tensor_rt_node]: [NitrosContext] Initializing application... [component_container_mt-1] [INFO] [1690282580.739824512] [tensor_rt_node]: [NitrosContext] Running application... [component_container_mt-1] [INFO] [1690282580.740747328] [tensor_rt_node]: [NitrosNode] Starting a heartbeat timer (eid=164) [component_container_mt-1] [INFO] [1690282581.187272768] [unet_decoder_node]: [NitrosNode] Starting post negotiation setup [component_container_mt-1] [INFO] [1690282581.189636608] [unet_decoder_node]: [NitrosNode] Getting data format negotiation results [component_container_mt-1] [INFO] [1690282581.189814016] [unet_decoder_node]: [NitrosPublisher] Negotiation ended with no results [component_container_mt-1] [INFO] [1690282581.189941056] [unet_decoder_node]: [NitrosPublisher] Use only the compatible publisher: topic_name="/unet/raw_segmentation_mask", data_format="nitros_image_mono8" [component_container_mt-1] [INFO] [1690282581.190038720] [unet_decoder_node]: [NitrosPublisher] Negotiation ended with no results [component_container_mt-1] [INFO] [1690282581.190119104] [unet_decoder_node]: [NitrosPublisher] Use only the compatible publisher: topic_name="/unet/colored_segmentation_mask", data_format="nitros_image_rgb8" [component_container_mt-1] [INFO] [1690282581.190212512] [unet_decoder_node]: [NitrosSubscriber] Use the negotiated data format: "nitros_tensor_list_nhwc_rgb_f32" [component_container_mt-1] [INFO] [1690282581.190394368] [unet_decoder_node]: [NitrosNode] Exporting the final graph based on the negotiation results [component_container_mt-1] [INFO] [1690282581.217646176] [unet_decoder_node]: [NitrosNode] Wrote the final top level YAML graph to "/home/badal/vslam_ws/install/isaac_ros_unet/share/isaac_ros_unet/JDKKGCNCOB.yaml" [component_container_mt-1] [INFO] [1690282581.218106880] [unet_decoder_node]: [NitrosNode] Calling user's pre-load-graph callback [component_container_mt-1] [INFO] [1690282581.218456000] [unet_decoder_node]: [NitrosNode] Loading application [component_container_mt-1] [INFO] [1690282581.218814016] [unet_decoder_node]: [NitrosContext] Loading application: '/home/badal/vslam_ws/install/isaac_ros_unet/share/isaac_ros_unet/JDKKGCNCOB.yaml' [component_container_mt-1] [INFO] [1690282581.237629792] [unet_decoder_node]: [NitrosNode] Linking Nitros pub/sub to the loaded application [component_container_mt-1] [INFO] [1690282581.238782848] [unet_decoder_node]: [NitrosNode] Calling user's post-load-graph callback [component_container_mt-1] [INFO] [1690282581.239325792] [unet_decoder_node]: [NitrosContext] Initializing application... [component_container_mt-1] [INFO] [1690282581.298128160] [unet_decoder_node]: [NitrosContext] Running application... [component_container_mt-1] [INFO] [1690282581.299689152] [unet_decoder_node]: [NitrosNode] Starting a heartbeat timer (eid=208) [component_container_mt-1] 2023-07-25 18:56:27.239 ERROR /home/badal/vslam_ws/src/isaac_ros_dnn_inference/isaac_ros_tensor_rt/gxf/tensor_rt/tensor_rt_inference.cpp@148: TRT ERROR: 3: Cannot find binding of given name: input_2:0 [component_container_mt-1] 2023-07-25 18:56:27.239 ERROR /home/badal/vslam_ws/src/isaac_ros_dnn_inference/isaac_ros_tensor_rt/gxf/tensor_rt/tensor_rt_inference.cpp@341: Failed to get binding index for input input_2:0 in model /home/badal/data/new_engine.plan [component_container_mt-1] 2023-07-25 18:56:27.242 WARN gxf/std/greedy_scheduler.cpp@241: Error while executing entity 164 named 'JROAGQYGZR_inference': GXF_FAILURE [component_container_mt-1] 2023-07-25 18:56:27.242 ERROR gxf/std/entity_executor.cpp@540: Entity [JROAGQYGZR_inference] must be in Lifecycle::kStarted or Lifecycle::kIdle stage before stopping. Current state is StartPending [component_container_mt-1] 2023-07-25 18:56:27.741 ERROR gxf/std/entity_executor.cpp@203: Entity with eid 164 not found! [component_container_mt-1] [WARN] [1690282587.741622848] [tensor_rt_node]: [NitrosNode] The heartbeat entity (eid=164) was stopped. The graph may have been terminated. [component_container_mt-1] [INFO] [1690282587.766059744] [unet_decoder_node]: [NitrosNode] Terminating the running application [component_container_mt-1] [INFO] [1690282587.766205152] [unet_decoder_node]: [NitrosContext] Interrupting GXF... [component_container_mt-1] [INFO] [1690282587.768190176] [unet_decoder_node]: [NitrosContext] Waiting on GXF... [component_container_mt-1] [INFO] [1690282587.768323328] [unet_decoder_node]: [NitrosContext] Deinitializing... [component_container_mt-1] [INFO] [1690282587.774302048] [unet_decoder_node]: [NitrosContext] Destroying context [component_container_mt-1] [INFO] [1690282587.775050144] [unet_decoder_node]: [NitrosNode] Application termination done [component_container_mt-1] [INFO] [1690282587.836418048] [tensor_rt_node]: [NitrosNode] Terminating the running application [component_container_mt-1] [INFO] [1690282587.836540736] [tensor_rt_node]: [NitrosContext] Interrupting GXF... [component_container_mt-1] [INFO] [1690282587.836605888] [tensor_rt_node]: [NitrosContext] Waiting on GXF... [component_container_mt-1] 2023-07-25 18:56:27.836 ERROR gxf/std/program.cpp@497: wait failed. Deactivating... [component_container_mt-1] 2023-07-25 18:56:27.838 ERROR gxf/core/runtime.cpp@1251: Graph wait failed with error: GXF_FAILURE [component_container_mt-1] [ERROR] [1690282587.838750656] [tensor_rt_node]: [NitrosContext] GxfGraphWait Error: GXF_FAILURE [component_container_mt-1] [INFO] [1690282587.838802848] [tensor_rt_node]: [NitrosNode] Application termination done [component_container_mt-1] [INFO] [1690282587.869010048] [dnn_image_encoder]: [NitrosNode] Terminating the running application [component_container_mt-1] [INFO] [1690282587.869272576] [dnn_image_encoder]: [NitrosContext] Interrupting GXF... [component_container_mt-1] [INFO] [1690282587.869350048] [dnn_image_encoder]: [NitrosContext] Waiting on GXF... [component_container_mt-1] [INFO] [1690282587.872316224] [dnn_image_encoder]: [NitrosContext] Deinitializing... [component_container_mt-1] [INFO] [1690282587.907488512] [dnn_image_encoder]: [NitrosContext] Destroying context [ERROR] [component_container_mt-1]: process has died [pid 4770, exit code -11, cmd '/home/badal/ros2_humble/install/rclcpp_components/lib/rclcpp_components/component_container_mt --ros-args -r __node:=unet_container -r __ns:=/'].

BADAL244 avatar Jul 28 '23 08:07 BADAL244

Could you confirm the input binding names of your engine file are correct? The way you are invoking the launch file is correct the launch arguments are passed through correctly.

swapnesh-wani-nvidia avatar Aug 07 '23 23:08 swapnesh-wani-nvidia

I have an engine file , i have confirmed the input and output binding name its correct . now the problem is my engine output is in the NCHW format for 19 classes. so i have a output of 1 * 19 * 224 * 224. but the IRIS (isaac ros image segmentation ) even after changing the "output_tensor_formats" default nitros_tensor_list_nhwc_rgb_f32 to nitros_list_nchw_rgb_f32 giving me the error, if i pass the default shape , then the height becomes 19 and classes becomes 224. so can you please tell me how to resolve this issue.

BADAL244 avatar Aug 11 '23 05:08 BADAL244

The code in Isaac ROS Image Segmentation/Unet Decoder is expecting the tensor output from inference on a UNet-architecture in NHWC format and does not generalize on a different format quite yet. You're using peoplesemsegnet_shuffleseg, however, which should output in NHWC as expected. Are you reading a different endpoint of the inference as "output" or something else?

hemalshahNV avatar Aug 11 '23 13:08 hemalshahNV

I am not using peopleseg ,I am using segformer from NGC catalog pretrained on cityscape dataset. I converted that into trt engine and passing it through the IRIS.

BADAL244 avatar Aug 11 '23 13:08 BADAL244

IRIS does not support SegFormer-architecture models yet but we have it on our radar. The modifications to have UNet Decoder support SegFormer at first glance do not seem significant so you could update it yourself. In the meantime, we'll get this on our roadmap.

hemalshahNV avatar Aug 11 '23 13:08 hemalshahNV

hello,

I am facing the exact issue, but for detection. and the weight is a custom one.

Did you solve it?

thank you

AbdullahGM1 avatar Nov 22 '23 06:11 AbdullahGM1

You have to pass your input output binding name at the time of conversion in onnx format. Then pass the same name for the input and output binding in this launch file then this will work.

BADAL244 avatar Nov 22 '23 09:11 BADAL244

from where do I get the name of the binding input and output?

it's not clear to me, if you can explain more, I would appreciate it

AbdullahGM1 avatar Nov 22 '23 09:11 AbdullahGM1

Input to the model

x = torch.randn(batch_size, 1, 224, 224, requires_grad=True) torch_out = torch_model(x)

Export the model

torch.onnx.export(torch_model, # model being run x, # model input (or a tuple for multiple inputs) "super_resolution.onnx", # where to save the model (can be a file or file-like object) export_params=True, # store the trained parameter weights inside the model file opset_version=10, # the ONNX version to export the model to do_constant_folding=True, # whether to execute constant folding for optimization input_names = ['input'], # the model's input names output_names = ['output'], # the model's output names dynamic_axes={'input' : {0 : 'batch_size'}, # variable length axes 'output' : {0 : 'batch_size'}})

BADAL244 avatar Nov 22 '23 09:11 BADAL244

Input names and output names is the binding name here

BADAL244 avatar Nov 22 '23 09:11 BADAL244

Thanks @BADAL244 for your help!

@AbdullahGM1 , please let us know if this fixes your issue so that we can close this one out.

jaiveersinghNV avatar Nov 27 '23 23:11 jaiveersinghNV

I wanted to add a quick addition to this based on my recent experience.

  • I recently trained a UNet model using TAO toolkit 5.0, which now allows you to export your model as an ONNX file.
  • I then ran /usr/src/tensorrt/bin/trtexec --onnx=f1tenth_unet.onnx --saveEngine=f1tenth_unet.engine to convert my model into a TensorRT engine like usual (according to this documentation).
  • Then I get to the step where I want to run this image segmentation model in ROS using an Isaac ROS node, so I headed over to the Isaac ROS image segmentation page. Specifically, I want to use the isaac_ros_unet_tensor_rt.launch.py file, as I have a TensorRT engine for my UNet model.
  • My initial launch command looks like this: ros2 launch isaac_ros_unet isaac_ros_unet_tensor_rt.launch.py input_image_width:=640 input_image_height:=480 network_image_width:=320 network_image_height:=320 engine_file_path:=/workspaces/isaac_ros-dev/src/isaac_ros_image_segmentation/models/f1tenth_unet.engine network_output_type:=argmax mask_height:=480 mask_width:=640 input_binding_names:=['input_2:0'] output_binding_names:=['argmax_1'] network_output_type:='argmax', based on the example.
  • However, when I ran that, the TensorRT node was complaining that "TRT ERROR: 3: Cannot find binding of given name: input_2:0." -- which is the problem we're talking about above.
  • This sounded like I had the wrong binding, so I spent some time trying to look for utilities to examine the structure of a TensorRT engine, and eventually stumbled upon this thread, which pointed me to NVIDIA's polygraphy package. This tool is probably the most convenient way to figure out what the names of the bindings within your TensorRT engine are. This tool should REALLY be referenced in the Isaac ROS or Deepstream documentation somewhere, as if we're deploying our models, we shouldn't have to search all over creation for some random utility. Not cool :,)
  • Once you have it installed, you can inspect your TensorRT engine file using the steps in this example. Step 2 is really what you're interested in if you're just looking at a TensorRT engine.
  • I ran polygraphy inspect model f1tenth_unet.engine --show layers and saw the following at the top of the output:

[I] ==== TensorRT Engine ==== Name: Unnamed Network 0 | Explicit Batch Engine
---- 1 Engine Input(s) ---- {input_1:0 [dtype=float32, shape=(1, 3, 320, 320)]}
---- 1 Engine Output(s) ---- {argmax_1 [dtype=int32, shape=(1, 320, 320, 1)]}

  • There's my answer: I see the engine's input binding is named input_1:0.
  • With that, I updated my launch command (replaced 'input_2:0' with 'input_1:0') and finally got my UNet model working.

That was way beyond the scope of your question, but I hope the context helps a bit @AbdullahGM1

nlitz88 avatar Dec 17 '23 02:12 nlitz88