Out-of-memory when running deepFilter on long audio files

[erykwieliczko]

Download (packed with xz) or create a 1 hour empty .wav file.

When trying to denoise it:

time CUDA_VISIBLE_DEVICES=1 deepFilter /tmp/silence.wav 
/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/df/io.py:9: UserWarning: `torchaudio.backend.common.AudioMetaData` has been moved to `torchaudio.AudioMetaData`. Please update the import path.
  from torchaudio.backend.common import AudioMetaData
2024-04-02 09:13:20 | INFO     | DF | Running on torch 2.2.2+cu121
2024-04-02 09:13:20 | INFO     | DF | Running on host d
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
2024-04-02 09:13:20 | INFO     | DF | Loading model settings of DeepFilterNet3
2024-04-02 09:13:20 | INFO     | DF | Using DeepFilterNet3 model at /home/ai/.cache/DeepFilterNet/DeepFilterNet3
2024-04-02 09:13:20 | INFO     | DF | Initializing model `deepfilternet3`
2024-04-02 09:13:20 | INFO     | DF | Found checkpoint /home/ai/.cache/DeepFilterNet/DeepFilterNet3/checkpoints/model_120.ckpt.best with epoch 120
2024-04-02 09:13:20 | INFO     | DF | Running on device cuda:0
2024-04-02 09:13:20 | INFO     | DF | Model loaded
2024-04-02 09:13:22 | WARNING  | DF | Audio sampling rate does not match model sampling rate (44100, 48000). Resampling...
/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/df/io.py:106: UserWarning: "sinc_interpolation" resampling method name is being deprecated and replaced by "sinc_interp_hann" in the next release. The default behavior remains unchanged.
  return ta_resample(audio, orig_sr, new_sr, **params)
Traceback (most recent call last):
  File "/home/somewhere/DeepFilterNet/env/bin/deepFilter", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/df/enhance.py", line 378, in run
    main(args)
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/df/enhance.py", line 78, in main
    audio = enhance(
            ^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/df/enhance.py", line 235, in enhance
    enhanced = model(spec.clone(), erb_feat, spec_feat)[0].cpu()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/df/deepfilternet3.py", line 411, in forward
    e0, e1, e2, e3, emb, c0, lsnr = self.enc(feat_erb, feat_spec)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/df/deepfilternet3.py", line 177, in forward
    c0 = self.df_conv0(feat_spec)  # [B, C, T, Fc]
         ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/container.py", line 217, in forward
    input = module(input)
            ^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 460, in forward
    return self._conv_forward(input, self.weight, self.bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/somewhere/DeepFilterNet/env/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 456, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Expected canUse32BitIndexMath(input) && canUse32BitIndexMath(output) to be true, but got false.  (Could this error message be improved?  If so, please report an enhancement request to PyTorch.)

real    0m8.745s
user    0m7.120s
sys     0m5.144s

On another terminal nvidia-smi skyrockets to 20GB and then crashes due to OOM. Doesn't happen with short files.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[erykwieliczko]

If you're facing the same problem, just use https://github.com/m-bain/whisperX

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 7 days.