Kerry Zhu

I tried to use rope with this command: ``` dora --verbose run \ solver=musicgen/musicgen_base_32khz_test \ model/lm/model_scale=xsmall \ conditioner=none \ transformer_lm.positional_embedding=sin_rope \ --clear ``` this is the configuration file musicgen_base_32khz_test: ```...

Hi, I am training a model with Audiocraft. In the training logs, I noticed that starting from epoch 6, many training summaries report grad_norm=INF or grad_norm=NAN. I also observed that...