liulurong comments

Repositories
Issues
Comments

Results 2 comments of


                                            liulurong

[SGLang Async Rollout] CUDA error: an illegal memory access was encountered

I also have a "illegal memory access" problem when using VerlEngine(backend="server") i.e. HttpServerEngineAdapter sglang 0.4.6.post4 > export GLOO_SOCKET_IFNAME=bond1 > export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 > MODEL=Qwen/Qwen2.5-0.5B-Instruct > python3 -m verl.trainer.main_ppo \ > algorithm.adv_estimator=grpo...

[SGLang Async Rollout] CUDA error: an illegal memory access was encountered

full error log ``` (TaskRunner pid=845875) {'actor_rollout_ref': {'actor': {'checkpoint': {'contents': ['model', (TaskRunner pid=845875) 'optimizer', (TaskRunner pid=845875) 'extra']}, (TaskRunner pid=845875) 'clip_ratio': 0.2, (TaskRunner pid=845875) 'clip_ratio_c': 3.0, (TaskRunner pid=845875) 'clip_ratio_high': 0.2, (TaskRunner...