segan icon indicating copy to clipboard operation
segan copied to clipboard

load failed when running train_segan.sh

Open andreaschandra opened this issue 6 years ago • 6 comments

RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa Parsed arguments: {'deconv_type': 'deconv', 'd_label_smooth': 0.25, 'z_depth': 256, 'preemph': 0.95, 'seed': 111, 'init_noise_std': 0.0, 'synthesis_path': 'dwavegan_samples', 'e2e_dataset': 'data/segan.tfrecords', 'save_freq': 50, 'g_type': 'ae', 'l1_remove_epoch': 150, 'epoch': 86, 'bias_D_conv': True, 'save_path': 'segan_allbiased_preemph', 'test_wav': None, 'g_learning_rate': 0.0002, 'unrolled_lstm': False, 'z_dim': 256, 'batch_size': 50, 'bias_downconv': True, 'denoise_epoch': 5, 'canvas_size': 16384, 'noise_decay': 0.7, 'g_nl': 'prelu', 'beta_1': 0.5, 'd_learning_rate': 0.0002, 'denoise_lbound': 0.01, 'bias_deconv': True, 'weights': None, 'model': 'gan', 'save_clean_path': 'test_clean_results', 'init_l1_weight': 100.0} Using device: /cpu:0 Creating GAN model *** Applying pre-emphasis of 0.95 *** *** Building Generator *** Biasing downconv in G Downconv (50, 16384, 1) -> (50, 8192, 16) Adding skip connection downconv 0 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 8192, 16) -> (50, 4096, 32) Adding skip connection downconv 1 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 4096, 32) -> (50, 2048, 32) Adding skip connection downconv 2 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 2048, 32) -> (50, 1024, 64) Adding skip connection downconv 3 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 1024, 64) -> (50, 512, 64) Adding skip connection downconv 4 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 512, 64) -> (50, 256, 128) Adding skip connection downconv 5 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 256, 128) -> (50, 128, 128) Adding skip connection downconv 6 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 128, 128) -> (50, 64, 256) Adding skip connection downconv 7 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 64, 256) -> (50, 32, 256) Adding skip connection downconv 8 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 32, 256) -> (50, 16, 512) Adding skip connection downconv 9 -- Enc: prelu activation -- Biasing downconv in G Downconv (50, 16, 512) -> (50, 8, 1024) -- Enc: prelu activation -- g_dec_depths: [512, 256, 256, 128, 128, 64, 64, 32, 32, 16, 1] -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 8, 2048) -> (50, 16, 512) -- Dec: prelu activation -- Fusing skip connection of shape (50, 16, 512) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 16, 1024) -> (50, 32, 256) -- Dec: prelu activation -- Fusing skip connection of shape (50, 32, 256) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 32, 512) -> (50, 64, 256) -- Dec: prelu activation -- Fusing skip connection of shape (50, 64, 256) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 64, 512) -> (50, 128, 128) -- Dec: prelu activation -- Fusing skip connection of shape (50, 128, 128) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 128, 256) -> (50, 256, 128) -- Dec: prelu activation -- Fusing skip connection of shape (50, 256, 128) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 256, 256) -> (50, 512, 64) -- Dec: prelu activation -- Fusing skip connection of shape (50, 512, 64) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 512, 128) -> (50, 1024, 64) -- Dec: prelu activation -- Fusing skip connection of shape (50, 1024, 64) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 1024, 128) -> (50, 2048, 32) -- Dec: prelu activation -- Fusing skip connection of shape (50, 2048, 32) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 2048, 64) -> (50, 4096, 32) -- Dec: prelu activation -- Fusing skip connection of shape (50, 4096, 32) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 4096, 64) -> (50, 8192, 16) -- Dec: prelu activation -- Fusing skip connection of shape (50, 8192, 16) -- Transposed deconvolution type -- Biasing deconv in G Deconv (50, 8192, 32) -> (50, 16384, 1) -- Dec: tanh activation -- Amount of alpha vectors: 21 Amount of skip connections: 10 Last wave shape: (50, 16384, 1)


num of G returned: 23 *** Discriminator summary *** D block 0 input shape: (50, 16384, 2) *** biasing D conv *** downconved shape: (50, 8192, 16) *** Applying VBN *** Applying Lrelu *** D block 1 input shape: (50, 8192, 16) *** biasing D conv *** downconved shape: (50, 4096, 32) *** Applying VBN *** Applying Lrelu *** D block 2 input shape: (50, 4096, 32) *** biasing D conv *** downconved shape: (50, 2048, 32) *** Applying VBN *** Applying Lrelu *** D block 3 input shape: (50, 2048, 32) *** biasing D conv *** downconved shape: (50, 1024, 64) *** Applying VBN *** Applying Lrelu *** D block 4 input shape: (50, 1024, 64) *** biasing D conv *** downconved shape: (50, 512, 64) *** Applying VBN *** Applying Lrelu *** D block 5 input shape: (50, 512, 64) *** biasing D conv *** downconved shape: (50, 256, 128) *** Applying VBN *** Applying Lrelu *** D block 6 input shape: (50, 256, 128) *** biasing D conv *** downconved shape: (50, 128, 128) *** Applying VBN *** Applying Lrelu *** D block 7 input shape: (50, 128, 128) *** biasing D conv *** downconved shape: (50, 64, 256) *** Applying VBN *** Applying Lrelu *** D block 8 input shape: (50, 64, 256) *** biasing D conv *** downconved shape: (50, 32, 256) *** Applying VBN *** Applying Lrelu *** D block 9 input shape: (50, 32, 256) *** biasing D conv *** downconved shape: (50, 16, 512) *** Applying VBN *** Applying Lrelu *** D block 10 input shape: (50, 16, 512) *** biasing D conv *** downconved shape: (50, 8, 1024) *** Applying VBN *** Applying Lrelu *** discriminator deconved shape: (50, 8, 1024) discriminator output shape: (50, 1)


Not clipping D weights Initializing optimizers... Initializing variables... Sampling some wavs to store sample references... sample noisy shape: (50, 16384) sample wav shape: (50, 16384) sample z shape: (50, 8, 1024) total examples in TFRecords data/segan.tfrecords: 48650 Batches per epoch: 973 [*] Reading checkpoints... [!] Load failed ./train_segan.sh: line 20: 13602 Killed CUDA_VISIBLE_DEVICES="" python main.py --init_noise_std 0. --save_path segan_allbiased_preemph --init_l1_weight 100. --batch_size 50 --g_nl prelu --save_freq 50 --preemph 0.95 --epoch 86 --bias_deconv True --bias_downconv True --bias_D_conv True

andreaschandra avatar Jun 22 '18 08:06 andreaschandra

i have this problem ,can you solve it

bailiangze avatar Dec 13 '18 13:12 bailiangze

i have this problem ,can you solve it

yes, i run it with smaller dataset, and if you just want to test the weight that has been train, you just attach it to the script for clean_wav.sh

andreaschandra avatar Dec 15 '18 12:12 andreaschandra

thank your reply。your answer help me solve this problem。Do you have a prtrained model ,because I do not have GPU.this trainning will spend me a lot of time。can you give me a pretrained model ; my tf is 1.9version.. I find the author‘s model is incompatile with me

bailiangze avatar Dec 15 '18 12:12 bailiangze

I see, but I was also using pretrained model from the author. I computed on Linux 16.10 and using the library version from requirements. you can downgrade the tensorflow version and make virtual environment for it

andreaschandra avatar Dec 16 '18 06:12 andreaschandra

Have you successfully replicated the experiment?I find that the g_adv loss is always 1 in training.And the loss seems to be different from the picture provided by the author.Can you show me the g_loss where you trained? My e-mail:[email protected]

fengqiyun avatar Mar 04 '19 15:03 fengqiyun

Could someone tell me where I can find the pretrained model? Thanks!

fycheng-code avatar Jul 18 '19 03:07 fycheng-code