IsaacLab icon indicating copy to clipboard operation
IsaacLab copied to clipboard

[Bug Report] Crash Training Ant on Conda/Ubuntu 24

Open 81578823 opened this issue 2 months ago • 2 comments

When I run the test task: ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task=Isaac-Ant-v0 --headless

The errors occur in my terminal: 2025-11-04T09:11:20Z [22,195ms] [Fatal] [carb.crashreporter-breakpad.plugin] 036: python!builtin_exec+0x14d (/usr/local/src/conda/python-3.11.14/Python/bltinmodule.c:1108) 2025-11-04T09:11:20Z [22,340ms] [Fatal] [carb.crashreporter-breakpad.plugin] 037: python!cfunction_vectorcall_FASTCALL_KEYWORDS+0x4f (/usr/local/src/conda/python-3.11.14/Include/internal/pycore_ceval.h:123) 2025-11-04T09:11:20Z [22,489ms] [Fatal] [carb.crashreporter-breakpad.plugin] 038: python!_PyEval_EvalFrameDefault.cold+0x19b9 (/usr/local/src/conda/python-3.11.14/Python/ceval.c:7321 (discriminator 1)) 2025-11-04T09:11:20Z [22,638ms] [Fatal] [carb.crashreporter-breakpad.plugin] 039: python!_PyFunction_Vectorcall+0x165 (/usr/local/src/conda/python-3.11.14/Python/frame.c:129) 2025-11-04T09:11:21Z [22,786ms] [Fatal] [carb.crashreporter-breakpad.plugin] 040: python!object_vacall+0x206 (/usr/local/src/conda/python-3.11.14/Include/internal/pycore_pyerrors.h:23) 2025-11-04T09:11:21Z [22,934ms] [Fatal] [carb.crashreporter-breakpad.plugin] 041: python!PyObject_CallMethodObjArgs+0x118 (/usr/local/src/conda/python-3.11.14/Include/object.h:537) 2025-11-04T09:11:21Z [23,079ms] [Fatal] [carb.crashreporter-breakpad.plugin] 042: python!PyImport_ImportModuleLevelObject+0x791 (/usr/local/src/conda/python-3.11.14/Python/import.c:1748) 2025-11-04T09:11:21Z [23,223ms] [Fatal] [carb.crashreporter-breakpad.plugin] 043: python!_PyEval_EvalFrameDefault+0x6419 (/usr/local/src/conda/python-3.11.14/Python/ceval.c:7428) 2025-11-04T09:11:21Z [23,371ms] [Fatal] [carb.crashreporter-breakpad.plugin] 044: python!_PyEval_Vector+0x145 (/usr/local/src/conda/python-3.11.14/Include/internal/pycore_ceval.h:73) 2025-11-04T09:11:21Z [23,517ms] [Fatal] [carb.crashreporter-breakpad.plugin] 045: python!PyEval_EvalCode+0x9d (/usr/local/src/conda/python-3.11.14/Python/ceval.c:1149) 2025-11-04T09:11:21Z [23,663ms] [Fatal] [carb.crashreporter-breakpad.plugin] 046: python!builtin_exec+0x14d (/usr/local/src/conda/python-3.11.14/Python/bltinmodule.c:1108) 2025-11-04T09:11:22Z [23,814ms] [Fatal] [carb.crashreporter-breakpad.plugin] 047: python!cfunction_vectorcall_FASTCALL_KEYWORDS+0x4f (/usr/local/src/conda/python-3.11.14/Include/internal/pycore_ceval.h:123) ^C2025-11-04T09:11:22Z [23,947ms] [Fatal] [carb.crashreporter-breakpad.plugin] 048: python!_PyEval_EvalFrameDefault.cold+0x19b9 2025-11-04T09:11:22Z [24,096ms] [Fatal] [carb.crashreporter-breakpad.plugin] 049: python!_PyFunction_Vectorc

System Info

Describe the characteristic of your environment:

  • Isaac Sim Version: 5.1.0
  • Isaaclab Version: 2.3.x
  • OS: Ubuntu 24.04
  • GPU: RTX 5080
  • CUDA: 12.8
  • GPU Driver: 570.195.03

81578823 avatar Nov 04 '25 09:11 81578823

Thank you for posting this. The team will review it.

RandomOakForest avatar Nov 06 '25 19:11 RandomOakForest

CPU: Intel ultra7-265K Memory: 16G*2 DDR5

81578823 avatar Nov 16 '25 17:11 81578823

I have figured it out finally!!! The interruption occurs since I use intel_pstate driver for my cpu(Intel ultra 7-265K), which is the newest CPU hardware having compatibility issues with ubuntu.

The solutions are as follows:

  • sudo gedit /etc/default/grub
  • Change the line GRUB_CMDLINE_LINUX_DEFAULT="quiet splash" to GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_pstate=disable" and save.
  • sudo update-grub
  • sudo reboot

After these operations, you can see the cpu driver has changed from intel_pstate to acpi-cpufreq according to cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_driver. The interruption does not occur any more.

ps: Do not forget to set CPU scaling_governor as schedutil, which will be better

81578823 avatar Nov 23 '25 12:11 81578823

@zoctipus for vis.

RandomOakForest avatar Dec 04 '25 22:12 RandomOakForest