Stage1 training crashes on eval - continued

[kruzel]

trafficstars

hi, thanks for a very interesting paper and supporting code. I'm trying to run training, but fail.

I tried to create the dataset few times, but it didn't help. dataset preparation looks ok, here are the results:

[kruzel]

succeeded to fix it by adding isnan checks in mGPT/metrics/utils.py in all scipy.linalg.norm inputs. if isnan return 0

[kruzel]

but now results looks like this, is that normal?

[Spark001]

same issue.

[Yashaswini-Srirangarajan]

Where do we have to place the HumanML3D dataset to get started on the training?

I am getting this error while training

[zybermonk]

succeeded to fix it by adding isnan checks in mGPT/metrics/utils.py in all scipy.linalg.norm inputs. if isnan return 0

Add the NAN check to the .npy files instead. Then remove them from all the folders and .txt files.

This should fix the issue.