[Bug Report] IsaacLab container does not work without UDP traffic

[5hadytru]

I have been successfully using the official Isaac Lab container ("nvcr.io/nvidia/isaac-lab:2.0.2") but have realized that building the container requires UDP traffic for something. When UDP traffic is not enabled (which is the case for, e.g, most of RunPod's datacenters) the container builds but the /workspace/isaaclab/ directory is empty. Is there a way to bypass the need for UDP traffic? Note that I believe disabled UDP traffic to be the issue since the RunPod support team came to this conclusion after a couple days of investigating; the container/template works in some datacenters and not others.

Steps to reproduce

1. Create RunPod template that pulls the isaac-lab image (set ACCEPT_EULA=Y and PRIVACY_CONSENT=Y in template env vars) and has /workspace/ as the volume mount point
2. Deploy a pod in any datacenter besides US-IL-1
3. Once the pod builds, use WebTerminal to ssh in (link provided by RunPod)
4. Observe empty /workspace/isaaclab directory, which is filled when UDP traffic is enabled

[RandomOakForest]

Thank you for posting this. Could you share here the RunPod template you are using or where is it coming from?

[5hadytru]

Here is the template I'm using

[TimPietrusky]

@5hadytru thanks for reporting this. i also tried to get this working on RunPod, but i'm not sure how to disable all UDP traffic for that image and if the image is even supposed to work when UDP is disabled.

@RandomOakForest maybe something that you can answer? will the image work without UDP and if yes, how can we deactivate anything in the image that needs UDP?

[RandomOakForest]

Thank you for following up. The team will review this.

[5hadytru]

Any progress on this?

[TimPietrusky]

@5hadytru we are working on UDP support, so hopefully this will just get rid of this problem (at least when running on RunPod)

[5hadytru]

@TimPietrusky any updates on this? Azure has officially become too expensive for me to continue

[LimePencil]

Any updates?