Byungsoo Oh

Fix local rank mismatch error when training on nodes with different number of GPUs

1

Fix #3408. Specifically, this PR fixes local rank mismatch error that occurs when training [Megatron-DeepSpeed](https://github.com/microsoft/Megatron-DeepSpeed) models on a cluster where **each node has different number of GPUs**. Note that this...

[BUG] Local rank mismatch error when training on nodes with different number of GPUs

1

**Describe the bug** When training Megatron-DeepSpeed models on nodes with different number of GPUs (e.g., node 0: 2 GPUs, node 1: 4 GPUs), local rank mismatch error (`AssertionError: expected local-rank...

Can DALI make the length of each sample uniform while decoding the audio file?

3

Hi, I would like to decode audio files (wav format) while making each sample have the uniform length. Specifically, I am trying to implement exactly the same operation as [`tf.audio.decode_wav()`](https://www.tensorflow.org/api_docs/python/tf/audio/decode_wav?hl=en)...