Huang Liankai

icon indicating a website link https://www.scut.edu.cn/

icon company South China University of Technology icon location South China University of Technology, Guangzhou, Guangdong, China, 510640 icon location South China University of Technology/major in Electronic Engineering

Results 3 comments of Huang Liankai

Assertion error regarding CUDA_VISIBLE_DEVICES encountered during training.

New issue: I get an error when I try to run with a single GPU: ``` ValueError: Please don't set ROCR_VISIBLE_DEVICES when HIP/CUDA_VISIBLE_DEVICES is set. ````

Assertion error regarding CUDA_VISIBLE_DEVICES encountered during training.

> Same issue when using sbatch, but fine with salloc🤔 Have you solved this problem? Not yet. Could you provide the script for running the training using `salloc`?

Assertion error regarding CUDA_VISIBLE_DEVICES encountered during training.

> ``` > salloc --partition=gpu_llm --gres=gpu:8 --time=3-00:00:00 --mem=1000G --cpus-per-task=8 /bin/bash > singularity exec --nv /public/home/.../verl_llamafactory_20250722.sif > bash examples/grpo_trainer/run_qwen2_5_vl-7b.sh > ``` > > ✅ > > ``` > #!/bin/bash > #SBATCH...