muntasir2000

` *** Starting training from scratch *** Training config: {'batch_size_per_gpu': 20, 'data_layer': , 'data_layer_params': {'augmentation': {'noise_level_max': -60, 'noise_level_min': -90, 'speed_perturbation_ratio': 0.1}, 'dataset_files': ['/hdd/stt-16k-seq2seq-train.csv'], 'input_type': 'spectrogram', 'max_duration': 16.7, 'num_audio_features': 160, 'shuffle':...

I also tried without using docker container. Anyways, I'll try using tensorflow:19.05-py3 image.

I tried using tensorflow:19.05-py3 docker image. Same issue. Training hangs Log file follows - -------------------------------- WARNING: Please update time_stretch_ratio to speed_perturbation_ratio WARNING: Please update time_stretch_ratio to speed_perturbation_ratio *** Building graph...

I tried to run nccl-tests, but the test also hangs the same way OpenSeq2Seq hangs. All GPUs show 100% usage constantly but hangs. I'm trying to follow this - https://github.com/NVIDIA/caffe/issues/10...

> @OliverMathias weight pruning and C++ infer. Can you share some more details on how you did weight pruning and C++ infer?