[Question] vkCreateFence failed with ERROR_OUT_OF_HOST_MEMORY

[LiliviaWu]

Question

---

🧩 Issue Summary

vkCreateFence failed with ERROR_OUT_OF_HOST_MEMORY occurs during runtime of an Isaac Sim–based Python simulation (carb.graphics-vulkan.plugin) even though system memory usage is low (< 10 GB of 64 GB available) and GPU memory is not fully utilized. The issue persists even after Vulkan ICD cleanup and verification.

---

🖥️ System Configuration

Component	Details
GPU	NVIDIA GeForce RTX 3080 Ti
Driver Version	550.163.01 (verified via nvidia-smi)
Vulkan API Version	1.3.277 (vulkaninfo)
CUDA Version	12.x (inferred from driver package)
CPU	Intel Core i9-12900K (includes integrated UHD 770)
OS	Ubuntu 22.04 LTS (x86-64, kernel 5.15+)
RAM	64 GB DDR5
Swap	2 GB
Python Environment	conda env: unitree_sim_env, Python 3.10, PyTorch 2.3+, Isaac Lab/Sim 2025
Display Server	X11
ICD configuration	/usr/share/vulkan/icd.d/nvidia_icd.json only (all Mesa/LLVM/AMDGPU ICDs removed)
Layers visible	nvidia_layers.json, VkLayer_MESA_device_select.json (disabled with NODEVICE_SELECT=1)

---

⚙️ Vulkan Diagnostic Output

GPU id = 0 (NVIDIA GeForce RTX 3080 Ti)
apiVersion = 1.3.277
driverVersion = 0x89a8c040
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU

All ICDs and layers verified via:

VK_LOADER_DEBUG=all vulkaninfo

Only NVIDIA ICD active.

---

🧪 Error Log Excerpt

[carb.graphics-vulkan.plugin] VkResult: ERROR_OUT_OF_HOST_MEMORY
[carb.graphics-vulkan.plugin] vkCreateFence failed with ERROR_OUT_OF_HOST_MEMORY (file descriptors: 304, soft limit: 2450, hard limit: 1048576)
[omni.kit.renderer.plugin] acquireNextFrameBufferNoWait: Failed to begin frame command list

Despite these, free -h shows:

Mem: 62Gi total, 7.5Gi used, 47Gi free, 7.8Gi buff/cache, 49–55Gi available
Swap: 2.0Gi total, 0 used

No spike in process RSS or mapped memory (VmSize ~100 GB, VmRSS ~3–4 GB constant).

---

🔍 Troubleshooting Already Performed

1. ✅ Verified correct ICD in /usr/share/vulkan/icd.d/nvidia_icd.json.

2. ✅ Removed/disabled all Mesa & LLVM ICDs (lvp_icd, radv_icd, intel_icd, VkLayer_MESA_device_select).

3. ✅ Verified Vulkan runtime (libvulkan.so.1) points to /lib/x86_64-linux-gnu/libvulkan.so.1.

4. ✅ No memory exhaustion (RAM, swap, GPU).

5. ✅ ulimit increased (ulimit -n 65536, ulimit -v unlimited).

6. ✅ Tested both headless (--no_render) and rendering modes — same error.

7. ✅ Vulkan validation layers show no vkAllocateMemory failure before the crash.

8. ✅ Kernel/dmesg logs clean (no OOM killer events).

9. ✅ NVIDIA driver modules (nvidia_uvm, nvidia_drm, nvidia_modeset) all loaded normally.

10. ⚠️ resource_tracker: There appear to be leaked shared_memory objects warnings from Python multiprocessing at shutdown (unclear relation).

---

Here is full log [run_full.log](https://github.com/user-attachments/files/22759541/run_full.log)

[LiliviaWu]

full_run.log

[RandomOakForest]

Thank you for posting this. It seems you may be hitting a hardware limitation, as the minimum spec. for Isaac Sim is GeForce RTX 4080. See this doc for ref..

[LiliviaWu]

Thank you for posting this. It seems you may be hitting a hardware limitation, as the minimum spec. for Isaac Sim is GeForce RTX 4080. See this doc for ref..

Thank you for your reply. However, it is Isaac Sim 4.5..

[LiliviaWu]

Thank you for posting this. It seems you may be hitting a hardware limitation, as the minimum spec. for Isaac Sim is GeForce RTX 4080. See this doc for ref..

Hi team — could you please take another look at this issue? It doesn’t seem to be a hardware limitation. For reference, version 4.5 runs fine on an RTX 3080 Ti, so the current problem is likely unrelated to insufficient specs. Thanks!

[jaxxstar]

Hi @LiliviaWu did you solve this issue?