Is it possible to run Jetpack 5.* images on Jetpack 4.* host?

[SkalskiP]

Hello. I'm trying to run nvcr.io/nvidia/l4t-pytorch:r34.1.0-pth1.12-py3 image on my Jetson AGX Xavier with Jetpack 4.5.1.

When I do:

docker run -it --rm --net=host --runtime nvidia nvcr.io/nvidia/l4t-pytorch:r34.1.0-pth1.12-py3

I get:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"process_linux.go:432: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: exec command: [/usr/bin/nvidia-container-cli --load-kmods configure --ldconfig=@/sbin/ldconfig.real --device=all --compute --compat32 --graphics --utility --video --display --pid=9735 /mnt/docker/data-root/overlay2/3279a31ce1fa3af1ce4a65a1c6ddb0229396bd2b97f17dbc61c919cbcedad2b8/merged]\\\\nnvidia-container-cli: mount error: file creation failed: /mnt/docker/data-root/overlay2/3279a31ce1fa3af1ce4a65a1c6ddb0229396bd2b97f17dbc61c919cbcedad2b8/merged/usr/lib/aarch64-linux-gnu/libcudnn_adv_infer_static_v8.a: file exists\\\\n\\\"\"": unknown.

When I run:

docker run -it --rm --net=host --runtime nvidia nvcr.io/nvidia/l4t-pytorch:r32.6.1-pth1.9-py3

Everything works just fine.

[dusty-nv]

Hi @SkalskiP, the JetPack 5.0 containers will only run on JetPack 5.x

[CourchesneA]

@dusty-nv follow-up question on this topic, is it possible to only build the images ? We have a jetson on jetpack 4.5 that we use as a CI which builds jetpack 5 images currently and it seems to be working. Should I expect problems ?

[dusty-nv]

We have a jetson on jetpack 4.5 that we use as a CI which builds jetpack 5 images currently and it seems to be working. Should I expect problems ?

I'm not sure, I haven't tried that. Do the images run okay on JetPack 5?

The JetPack 5 images are far more self-contained (in that CUDA/cuDNN/ect are installed inside the container instead of mounted from the host), which is probably why it builds okay on JetPack 4.x

[CourchesneA]

It is still early development, it seems okay for now. I am trying to figure what is still mounted from the host, it seems some libraries for GStreamer might still be mounted somehow. I got an issue where a few libs including /lib/aarch64-linux-gnu/tegra/libdrm.so.2 were empty. It seems to be fixed by adding --runtime nvidia or --gpus all

[dusty-nv]

It seems to be fixed by adding --runtime nvidia or --gpus all

Yes, these will mount those libraries. You can see which ones are mounted by checking the CSV files under /etc/nvidia-container-runtime/host-files-for-container.d/

[CourchesneA]

Awesome this will be really helpful, thanks !