Inference time jetson nano

[sieuwe1]

What would be expected inference time on a jetson nano? With 4 to 5 different depth levels.

Thanks

Sieuwe

[jankais3r]

For what it's worth, here are inference tests using the default configuration on Jetson Nano 2GB:

SCENEFLOW

CUDA_VISIBLE_DEVICES=0 python3 run_binary_depth_estimation.py \
    --arch bi3dnet_binary_depth \
    --bi3dnet_featnet_arch featextractnetspp \
    --bi3dnet_featnethr_arch featextractnethr \
    --bi3dnet_segnet_arch segnet2d \
    --bi3dnet_refinenet_arch segrefinenet \
    --featextractnethr_out_planes 16 \
    --segrefinenet_in_planes 17 \
    --segrefinenet_out_planes 8 \
    --crop_height 576 --crop_width 960 \
    --disp_vals 24 36 54 96 144 \
    --img_left  '../data/sf_img_left.jpg' \
    --img_right '../data/sf_img_right.jpg' \
    --pretrained '../model_weights/sf_binary_depth.pth.tar'

Measured with time: real 2m24.842s

KITTI15

CUDA_VISIBLE_DEVICES=0 python3 run_binary_depth_estimation.py \
    --arch bi3dnet_binary_depth \
    --bi3dnet_featnet_arch featextractnetspp \
    --bi3dnet_featnethr_arch featextractnethr \
    --bi3dnet_segnet_arch segnet2d \
    --bi3dnet_refinenet_arch segrefinenet \
    --featextractnethr_out_planes 16 \
    --segrefinenet_in_planes 17 \
    --segrefinenet_out_planes 8 \
    --crop_height 384 --crop_width 1248 \
    --disp_vals 12 21 30 39 48 \
    --img_left  '../data/kitti15_img_left.jpg' \
    --img_right '../data/kitti15_img_right.jpg' \
    --pretrained '../model_weights/kitti15_binary_depth.pth.tar'

Measured with time: real 1m41.397s

Based on my experience Jetson Nano is not powerful enough to run Bi3D properly. The main issue is the lack of memory - you can't run the continuous mode at all as it gets killed by the system (examples above are from the quantized mode). It is possible that the 4GB model could run also the continuous mode.

[sieuwe1]

@jankais3r thanks for the performance results. Looks indeed like the jetson nano 2gb won't be able to do it. What amount of depth levels did you use and what resolution?

A gtx1080 would be definitely strong enough for continuous mode?

Thanks

Sieuwe

[jankais3r]

I don't have access to any other Nvidia GPU besides the Jetson Nano, but I tried to run Bi3D in CPU-only mode on my Macbook. I was able to generate a map with bi3dnet_continuous_depth_2D, and the process utilized up to 19.9GB of RAM at one point. I also tried to run bi3dnet_continuous_depth_3D, which used up to 33.5GB RAM and then my system ran out of memory and the process was killed by the OS, so I don't know how much more memory it would need. I assume that if you would run this on a GPU, large portion of this memory would come from the GPU and not the system RAM, but AFAIK all Jetsons have unified memory design, and based on this test none of them has enough memory to run this inference in full.

[EhrazImam]

Hi @jankais3r i am trying to use it in jetson xavier nx can you let me know which version of docker you are using as it is recommended 19.03.11 and I am having 20.10.7....while building docker i am having issue...

[jankais3r]

Hi, I cannot comment on the Docker approach, as I used Conda environment for my tests.