The question about training CLAP RVQ

[yangdongchao]

It seems that in the ClapRVQTrainer code, you donot use any gradient backward? How to understood this?

[zhvng]

Hey, thanks for your interest in this repo! The RVQ parameters are updated via ema. See this relevant thread

[yangdongchao]

Hey, thanks for your interest in this repo! The RVQ parameters are updated via ema. See this relevant thread

thanks for your reply. I want to ask how to judge the training is successful? When we can stop the training?

[zhvng]

I stopped when the eval loss stopped going down. seems to work good enough.

[yangdongchao]

I stopped when the eval loss stopped going down. seems to work good enough.

Thanks for your relpy. Lastly ,I want to ask, whether you only use audio to extract embedding and use RVQ to quantize it in the training proces. The text is only used in the inference stage?