Ziyang Huang
Ziyang Huang
> Sounds like it is indeed running the inference model. It will hang there till it actually completes its inference. If the GPU's are occupied that sounds like its doing...
> In my training attempt (full model fine tuning with llama on HH datasets), in SFT stage, the training loss is not going down. In DPO, all outputs are NAN....
FYI: I run `python search_r1_agent.py`, then `bash train.sh`. It hang like this:
It is just like this, no dialogue. I have not read the code, so i do not know what happended😂
@SiyunZhao Hi, I use agl v.0.1.2 to reproduce the search_r1. I use the env set in scripts/setup_stable_gpu.sh. But it kept throw errors like: Do you have any ideas on this...
Do you run successfully with multi nodes? @wendongbi