conditioned-u-net
conditioned-u-net copied to clipboard
Veleslavia
execute . ./train_uspec.sh show AttributeError: 'AttrDict' object has no attribute 'multitask'
I set up all environment then try to execute the command . ./train_uspec.sh show the error below:
conditioned-u-net/code$ . ./train_uspec.sh
INFO - unet - Running command 'main'
INFO - unet - Started run with ID "2"
ERROR - unet - Failed after 0:00:00!
Traceback (most recent calls WITHOUT Sacred internals):
File "train_uspec.py", line 27, in main
multitask=exp_config.multitask, conditioning=exp_config.conditioning,
AttributeError: 'AttrDict' object has no attribute 'multitask'
Then I check the code from utils.attr_dict import AttrDict but can't find the multitask config.
Have any advice how to solve this problem?
Thank you for reading my question.
Hello, you need to use a config file where these parameters are specified. Examples of configuration files can be found here https://drive.google.com/drive/folders/1Hv9JwPuFIZ4SMlp0jc2OB3PKctcBCdUw for each experiment described in the paper.
Kind regards, Olga
I add the multitask = False in config.py
And change the runs to checkpoints in action_pipeline.py
Then get the error below:
conditioned-u-net/code$ . ./train_uspec.sh
INFO - unet - Running command 'main'
INFO - unet - Started run with ID "5"
Loaded source: Bassoon
Loaded source: Cello
Loaded source: Clarinet
Loaded source: DoubleBass
Loaded source: Flute
Loaded source: Horn
Loaded source: Oboe
Loaded source: Saxophone
Loaded source: Trombone
Loaded source: Trumpet
Loaded source: Tuba
Loaded source: Viola
Loaded source: Violin
Loaded source: Bassoon
Loaded source: Cello
Loaded source: Clarinet
Loaded source: DoubleBass
Loaded source: Flute
Loaded source: Horn
Loaded source: Oboe
Loaded source: Saxophone
Loaded source: Trombone
Loaded source: Trumpet
Loaded source: Tuba
Loaded source: Viola
Loaded source: Violin
INFO - train_logger - Starting epoch 0/1000.
/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/sacred/stdout_capturing.py:184: UserWarning: tee_stdout.wait timeout. Forcibly terminating.
warnings.warn("tee_stdout.wait timeout. Forcibly terminating.")
/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/sacred/stdout_capturing.py:190: UserWarning: tee_stderr.wait timeout. Forcibly terminating.
warnings.warn("tee_stderr.wait timeout. Forcibly terminating.")
ERROR - unet - Failed after 0:00:03!
Traceback (most recent calls WITHOUT Sacred internals):
File "train_uspec.py", line 45, in main
pipeline.train_model()
File "/home/mtl/Desktop/Video_Focusing_Music_Separation/conditioned-u-net/code/action_pipeline.py", line 267, in train_model
self.run_phase(epoch)
File "/home/mtl/Desktop/Video_Focusing_Music_Separation/conditioned-u-net/code/action_pipeline.py", line 214, in run_phase
for num_it, (inputs, gt_data, *aux_data) in enumerate(self.loader):
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 628, in __next__
data = self._next_data()
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1333, in _next_data
return self._process_data(data)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1359, in _process_data
data.reraise()
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/_utils.py", line 543, in reraise
raise exception
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 58, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/mtl/Desktop/Video_Focusing_Music_Separation/conditioned-u-net/code/datasets/solos.py", line 109, in __getitem__
sample_idx = random.randint(0, len(self.data[source_name])-1)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/random.py", line 222, in randint
return self.randrange(a, b+1)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/random.py", line 200, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0,0, 0)
It seems that the dataset that you're using is empty. Please, make sure that you use one of the configs that I mentioned above and specify the folder where the data is located. You may need to pre-process the data before training the model.
I have download the training data Solos raw25, and set the path in train_uspec.sh:
CUDA_VISIBLE_DEVICES=0 python train_uspec.py with exp_config.dataset_dir='./Solos_raw25'
I'm curious about how to pre-process the dataset? It seems didn't mention in README.md.
Right. I'm not sure in which format the data come from but you surely need to
- Extract audio from video data - per folder, I believe
- Downsample the data to 11025 Hz
I have pre-processed all the folders, and change the config.py dataset_dir to my path dataset_dir='./Solos_raw25/videos',
But still get the same error message:
conditioned-u-net/code$ . ./train_uspec.sh
INFO - unet - Running command 'main'
INFO - unet - Started run with ID "8"
Loaded source: Bassoon
Loaded source: Cello
Loaded source: Clarinet
Loaded source: DoubleBass
Loaded source: Flute
Loaded source: Horn
Loaded source: Oboe
Loaded source: Saxophone
Loaded source: Trombone
Loaded source: Trumpet
Loaded source: Tuba
Loaded source: Viola
Loaded source: Violin
Loaded source: Bassoon
Loaded source: Cello
Loaded source: Clarinet
Loaded source: DoubleBass
Loaded source: Flute
Loaded source: Horn
Loaded source: Oboe
Loaded source: Saxophone
Loaded source: Trombone
Loaded source: Trumpet
Loaded source: Tuba
Loaded source: Viola
Loaded source: Violin
INFO - train_logger - Starting epoch 0/1000.
/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/sacred/stdout_capturing.py:184: UserWarning: tee_stdout.wait timeout. Forcibly terminating.
warnings.warn("tee_stdout.wait timeout. Forcibly terminating.")
/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/sacred/stdout_capturing.py:190: UserWarning: tee_stderr.wait timeout. Forcibly terminating.
warnings.warn("tee_stderr.wait timeout. Forcibly terminating.")
ERROR - unet - Failed after 0:00:03!
Traceback (most recent calls WITHOUT Sacred internals):
File "train_uspec.py", line 45, in main
pipeline.train_model()
File "/home/mtl/Desktop/Video_Focusing_Music_Separation/conditioned-u-net/code/action_pipeline.py", line 267, in train_model
self.run_phase(epoch)
File "/home/mtl/Desktop/Video_Focusing_Music_Separation/conditioned-u-net/code/action_pipeline.py", line 214, in run_phase
for num_it, (inputs, gt_data, *aux_data) in enumerate(self.loader):
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 628, in __next__
data = self._next_data()
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1333, in _next_data
return self._process_data(data)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1359, in _process_data
data.reraise()
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/_utils.py", line 543, in reraise
raise exception
ValueError: Caught ValueError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 58, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/mtl/Desktop/Video_Focusing_Music_Separation/conditioned-u-net/code/datasets/solos.py", line 109, in __getitem__
sample_idx = random.randint(0, len(self.data[source_name])-1)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/random.py", line 222, in randint
return self.randrange(a, b+1)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/random.py", line 200, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0,0, 0)
Hi! Folders should have the same names as specified in the Dataset class here https://github.com/Veleslavia/conditioned-u-net/blob/master/code/datasets/solos.py
Could you please check (when running your code) that the metadata and the data is loaded correctly? I would advise to put a breakpoint here: https://github.com/Veleslavia/conditioned-u-net/blob/master/code/datasets/solos.py#L57 and check self.meta and self.data for the dataset before running training. You may also need to disable multiprocessing to being able to do debug.
Seems my dataset didn't have complete data, let me check about my dataset. I'll try to debug as your advice, thank you very much!
I re-download the Solos dataset and pre-processed all the audio file into 11025Hz (Only 639 file due to download link been delete or setting to private):
Then try to train the code, it shows error below:
conditioned-u-net/code$ python train_uspec.py
INFO - unet - Running command 'main'
INFO - unet - Started run with ID "13"
Loaded source: Bassoon
Loaded source: Cello
Loaded source: Clarinet
Loaded source: DoubleBass
Loaded source: Flute
Loaded source: Horn
Loaded source: Oboe
Loaded source: Saxophone
Loaded source: Trombone
Loaded source: Trumpet
Loaded source: Tuba
Loaded source: Viola
Loaded source: Violin
Loaded source: Bassoon
Loaded source: Cello
Loaded source: Clarinet
Loaded source: DoubleBass
Loaded source: Flute
Loaded source: Horn
Loaded source: Oboe
Loaded source: Saxophone
Loaded source: Trombone
Loaded source: Trumpet
Loaded source: Tuba
Loaded source: Viola
Loaded source: Violin
INFO - train_logger - Starting epoch 0/1000.
/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/sacred/stdout_capturing.py:184: UserWarning: tee_stdout.wait timeout. Forcibly terminating.
warnings.warn("tee_stdout.wait timeout. Forcibly terminating.")
/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/sacred/stdout_capturing.py:190: UserWarning: tee_stderr.wait timeout. Forcibly terminating.
warnings.warn("tee_stderr.wait timeout. Forcibly terminating.")
ERROR - unet - Failed after 0:00:23!
Traceback (most recent calls WITHOUT Sacred internals):
File "train_uspec.py", line 45, in main
pipeline.train_model()
File "/home/mtl/Desktop/Video_Focusing_Music_Separation/conditioned-u-net/code/action_pipeline.py", line 267, in train_model
self.run_phase(epoch)
File "/home/mtl/Desktop/Video_Focusing_Music_Separation/conditioned-u-net/code/action_pipeline.py", line 214, in run_phase
for num_it, (inputs, gt_data, *aux_data) in enumerate(self.loader):
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 628, in __next__
data = self._next_data()
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1333, in _next_data
return self._process_data(data)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1359, in _process_data
data.reraise()
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/_utils.py", line 543, in reraise
raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
data = fetcher.fetch(index)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 61, in fetch
return self.collate_fn(data)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
return collate(batch, collate_fn_map=default_collate_fn_map)
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 143, in collate
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 143, in <listcomp>
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/home/mtl/anaconda3/envs/conditioned_unet/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 139, in collate
raise RuntimeError('each element in list of batch should be of equal size')
RuntimeError: each element in list of batch should be of equal size
I am able to do training with batch_size set to 1, but the loss doesn't converge. However, when I set the batch_size > 1, I get this RuntimeError: each element in list of batch should be of equal size error.
Could you provide any guidance or clue to what causes this error and how to resolve it?
Hi, I am able to run the training code with batch_size set to 16 (due to hardware constraint) after I commented out the returned item "aux_source_indices" in the "solos.py" file, and eventually get the checkpoint audio files. However, I opened these audio files but could not hear anything. In addition, could you provide the codes for the inference part for the purpose of testing the trained model?
Hello,
Unfortunately, I don't recall checkpointing audio files, I would rather checkpoint the model, or store some masks and/or masked spectrograms during the validation after each epoch (most likely to tensorboard). I don't think that you would be able to hear anything from the masks.
Once you have trained a model (or you can use a pretrained one as they are shared), you need to use code/test_uspec.py file for inference.
Kind regards, Olga
Thank you for your reply!
I tried to run the "test_uspec.py" file using the checkpoint files that I trained, and got the errors shown in the screenshot below:
For testing and debugging, I ran the same “tes_uspec.py” file with your pre-trained model and the config.json file from the "Exp. 10" folder using the same command as above:
CUDA_VISIBLE_DEVICES=0 python test_uspec.py with experiments_meta/10/config.json exp_config.model_checkpoint='00010/CP0990.pth'
The result produced at the end seems more like an evaluation of a Solos dataset rather than a prediction. Is this understanding correct?
What I intend to do is to input a piece of music to run the inference, and to get a prediction for 13 instruments whether or not presented in the music. Could you kindly provide subsequent help on how to do this?
It seems that I have to add the "if-statement" shown below to the "test_uspec.py" file to be able to run the test on sepc_unet, but it only works on the Solos dataset.
if exp_config.model == 'specunet':
model = Unet(use_dropout=exp_config.use_dropout, complementary=exp_config.complementary,
conditioning=exp_config.conditioning)
elif exp_config.model == 'mhunet':
model = MHUnet(use_dropout=exp_config.use_dropout, complementary=exp_config.complementary,
multitask=exp_config.multitask, conditioning=exp_config.conditioning,
use_bias=exp_config.use_bias, n_decoders=exp_config.num_decoders,
num_downs=exp_config.num_downblocks)
I tried to run the command CUDA_VISIBLE_DEVICES=0 python test_uspec.py with experiments_meta/10/config.json exp_config.model_checkpoint='00010/CP0990.pth' exp_config.dataset='urmp' exp_config.dataset_dir='./Datasets/urmp', but it did not seem to read the audio file and run the inference...
URMP dataset seems need to do some pre-process, could you give a hint?
Hi! I think you're right, and apparently I don't have access to the pre-processed dataset anymore. I added you to the dev repo for this project (please, accept the invite), you can find a corresponding data processor there: https://github.com/Veleslavia/vimss_torch_dev/blob/master/datasets/urmp_dp.py
From what I remember, you need _process_dataset function from there.
What I intend to do is to input a piece of music to run the inference, and to get a prediction for 13 instruments whether or not presented in the music. Could you kindly provide subsequent help on how to do this?
I think by design the scripts are expecting to have a ground truth to compute the metrics etc. One option is to prepare a compatible dumb ground truth masks, and then save intermediate predictions for all instruments.
Thank you for your reply!
I spent a week trying to figure out how to use the "urmp_dp.py" file to perform data pre-processing. I run the code using the command python urmp_dp.py parallel_data_processor --raw_data_dir=./Datasets/urmp --output_dir=./Datasets/urmpv2numpy, but I am still getting errors despite already resolved quite a few by modifying the code.
One of the errors I am having trouble to resolve is shown in the image below. The error message reads "all input arrays must have the same shape". I can't figured out why and which lines of the code is causing the error. Could it be the np.stack() or pool.map()?
I tried to print out the shape of some data, hope it will help to explain where I am stuck.