s4 icon indicating copy to clipboard operation
s4 copied to clipboard

Several examples doesn't work (Sashimi checkpoints / sampleRNN training)

Open RochMollero opened this issue 1 year ago • 4 comments

Hello,

So I've cloned the repo and downloaded the sashimi checkpoints on hugging face in order to reproduce them and apply the method on my dataset. I've found three main problems:

any audio generated with sashimi is basically noise i.e. :

this command : python -m generate experiment=audio/sashimi-youtubemix checkpoint_path=checkpoints/sashimi_youtubemix_v4.pt n_samples=10 l_sample=16000 load_data=false

leads to : sashimi generation.zip

=> It does work with SampleRNN though.

=> With Wavenet it doesn't work either python -m generate experiment=audio/wavenet-sc09 checkpoint_path=checkpoints/wavenet_sc09.pt n_samples=1 l_sample=16000 load_data=false gives the error :

module.py", line 2152, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for SequenceLightningModule:
	Missing key(s) in state_dict: "encoder.0.weight", "decoder.0.output_transform.weight", "decoder.0.output_transform.bias". 
	Unexpected key(s) in state_dict: "encoder.weight", "decoder.output_transform.weight", "decoder.output_transform.bias". 

Training with SampleRNN doesn't work on youtubemix and beethoven:

so first i've noticed there was a reorganisation probably so I had to modify some .yaml for the new architecture

+++ b/configs/experiment/audio/samplernn-youtubemix.yaml
@@ -6,7 +6,7 @@ defaults:
   - /task: multiclass_classification
   - /optimizer: adamw
   - /scheduler: plateau
-  - /model: samplernn
+  - /model: baseline/samplernn

but then it still don't work, with python -m train experiment=audio/samplernn i get

    assert self.val_sanity_progress_bar_id is not None
AssertionError

and if I deactivate RichProgressBar, then I can see that it's because val/loss is not logged and I absolutely can't understand why. I've tried hunting differences between youtubemix and sc09 onwhich the training starts, but to no success.

What happens ? I've seen the "kernel what changed" to V4: is this why examples on sashmi are not working ? Should I go back to v3 ? I use pykeops but not the extension/kernels, would that be a reason ?

Regards,

Edit : also python setup.py install is deprectaed, I'm not sure how i should install the new kernels

RochMollero avatar Nov 23 '23 18:11 RochMollero