Alexander Kaigorodov

Well, let me join this thread too.. Have the same issue as described above The code I run can be found here: https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-v4/train.py Configuration I use ``` { "zero_allow_untested_optimizer": True,...

I have solved my problem by choosing right combination of python version and packages versions. If someone is interested in it: - python 3.8 - torch==2.0.0 - deepspeed==0.9.1 - pytorch-lightning==1.9.1...

@maomao279 Have you tried v4neo? + are you sure you use the same versions during the run and which cuda version do you use (not sure the last is important,...