NVIDIA
GPU-based vectorized Specaug Version 2
What does this PR do ?
This PR proposes a faster version of the Spectrogram Augmentation module for time and frequency masking.
After this I think that we may also have to add the flag in the audio processing module: https://github.com/amorari-nvidia/NeMo/blob/d26eab45d54e9e00e74c68ea4b0bbbf7b48a7a50/nemo/collections/asr/modules/audio_preprocessing.py#L494
And also we may have to add a test, adding the pytest run_only_on('GPU') flag the same it is done for the numba code: #https://github.com/amorari-nvidia/NeMo/blob/d26eab45d54e9e00e74c68ea4b0bbbf7b48a7a50/tests/collections/asr/test_asr_modules.py#L71
@amorari-nvidia As a last thing, could you remove the numba specaug implementation altogether in this PR? IIRC it will automatically kick in with numba 0.58 (or when numba+cuda versions are compatible for earlier versions), overriding the implementation here. CC @titu1994 pls double check if I'm right about this.
You can remove it entirely, or set the default flag of numba cuda to false. I think that's sufficient for the time being.
If you can remove it cleanly, then that's fine too
I will just set the Numba version default flag to false for now
The Numba SpecAugment version is already set to false, while the vectorized version is set to true and is the default code.
