frechet_audio_distance: Instructions not working

[jonnor]

Hi. As is the instructions for frechet_audio_distance give errors:

(fad) [jon@jon-thinkpad google-research]$ python -m frechet_audio_distance.create_embeddings_main --input_files test_audio/test_files_background.cvs --stats stats/background_stats
.... Tensorflow nagging ...
Traceback (most recent call last):
  File "/home/jon/.conda/envs/fad/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/jon/.conda/envs/fad/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jon/work/senselab/soundquality/google-research/frechet_audio_distance/create_embeddings_main.py", line 26, in <module>
    from frechet_audio_distance import create_embeddings_beam
  File "/home/jon/work/senselab/soundquality/google-research/frechet_audio_distance/create_embeddings_beam.py", line 36, in <module>
    from frechet_audio_distance.audioset_model import AudioSetModel
  File "/home/jon/work/senselab/soundquality/google-research/frechet_audio_distance/audioset_model.py", line 25, in <module>
    from tensorflow_models.audioset import mel_features
ImportError: cannot import name 'mel_features'

This seems to be due to a change in Tensorflow models, where the vggish directory was moved. https://github.com/tensorflow/models/commit/4079c5d9693142a406f6ff392d14e2034b5f496d#diff-bdac73c3c32a5ba3834aa4ce2b8b345e

Using an earlier version seems to work fine, however then one also have to use Tensorflow 1.15

svn export https://github.com/tensorflow/models/tags/v1.13.0/research/audioset tensorflow_models/audioset

pip install tensorflow==1.15

Btw it also seems that Apache Beam supports Python3 now. Right now frechet_audio_distance is however not compatible. At least there are some trivial errors like use of xrange.

Would a PR which updates to Python3 and TensorFlow 2.x be welcomed?

[jonnor]

Any feedback on this?

[aframires]

I also had to do this myself. As google is not answering, i guess you can create your own version of the FAD. Will save us all lots of work!

[javiernistal]

You can find mel_features.py in tensorflow_models/audioset/vggish/. To fix this and oother import problems just create tensorflow_models/audioset/__init__.py and add the line: from .vggish import mel_features, vggish_params, vggish_slim Also, in file tensorflow_models/audioset/vggish/vggish_slim.py you should change the import to from . import vggish_params as params.

You will also have problems with Tensorflow version: for me it worked r1.15 and changing in file frechet_audio_distance/fad_utils.py line 37 to tf_record = tf.python_io.tf_record_iterator(filename).__next__()

[RussellSB]

Thanks

You can find mel_features.py in tensorflow_models/audioset/vggish/. To fix this and oother import problems just create tensorflow_models/audioset/__init__.py and add the line: from .vggish import mel_features, vggish_params, vggish_slim Also, in file tensorflow_models/audioset/vggish/vggish_slim.py you should change the import to from . import vggish_params as params.

You will also have problems with Tensorflow version: for me it worked r1.15 and changing in file frechet_audio_distance/fad_utils.py line 37 to tf_record = tf.python_io.tf_record_iterator(filename).__next__()

Really helped! Thanks

[qinyang39]

I am also stacking on this step, when I fix the import problem. I got core dumped error as below... (I am using tf 2.5.0) Anyone can help?

(fad-env) kp1@nipa2021-1226:~/google-research$ python -m frechet_audio_distance.create_embeddings_main --input_files test_audio/test_files_background.cvs --stats stats/background_stats 2021-07-23 16:52:30.124584: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory 2021-07-23 16:52:30.124631: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. I0723 16:52:31.658765 140422069139200 pipeline.py:186] Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. Fatal Python error: Segmentation fault Current thread 0x00007fb68f8cb700 (most recent call first): File "/usr/local/lib/python3.7/bz2.py", line 346 in decompress File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/internal/pickler.py", line 277 in loads File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 857 in init File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/core.py", line 1222 in init File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/core.py", line 1539 in FlatMap File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/core.py", line 1593 in Map File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/io/iobase.py", line 895 in expand File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 215 in apply_PTransform File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 185 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 694 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 601 in ror File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/io/textio.py", line 569 in expand File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 215 in apply_PTransform File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 185 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 694 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 651 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 601 in ror File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 1086 in ror File "/home/kp1/google-research/frechet_audio_distance/create_embeddings_beam.py", line 353 in create_pipeline File "/home/kp1/google-research/frechet_audio_distance/create_embeddings_main.py", line 69 in main File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/absl/app.py", line 258 in _run_main File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/absl/app.py", line 312 in run File "/home/kp1/google-research/frechet_audio_distance/create_embeddings_main.py", line 75 in File "/usr/local/lib/python3.7/runpy.py", line 85 in _run_code File "/usr/local/lib/python3.7/runpy.py", line 193 in _run_module_as_main Segmentation fault (core dumped)

[RussellSB]

I am also stacking on this step, when I fix the import problem. I got core dumped error as below... (I am using tf 2.5.0) Anyone can help?

(fad-env) kp1@nipa2021-1226:~/google-research$ python -m frechet_audio_distance.create_embeddings_main --input_files test_audio/test_files_background.cvs --stats stats/background_stats 2021-07-23 16:52:30.124584: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory 2021-07-23 16:52:30.124631: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. I0723 16:52:31.658765 140422069139200 pipeline.py:186] Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. Fatal Python error: Segmentation fault Current thread 0x00007fb68f8cb700 (most recent call first): File "/usr/local/lib/python3.7/bz2.py", line 346 in decompress File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/internal/pickler.py", line 277 in loads File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 857 in init File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/core.py", line 1222 in init File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/core.py", line 1539 in FlatMap File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/core.py", line 1593 in Map File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/io/iobase.py", line 895 in expand File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 215 in apply_PTransform File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 185 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 694 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 601 in ror File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/io/textio.py", line 569 in expand File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 215 in apply_PTransform File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 185 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 694 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/pipeline.py", line 651 in apply File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 601 in ror File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/apache_beam/transforms/ptransform.py", line 1086 in ror File "/home/kp1/google-research/frechet_audio_distance/create_embeddings_beam.py", line 353 in create_pipeline File "/home/kp1/google-research/frechet_audio_distance/create_embeddings_main.py", line 69 in main File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/absl/app.py", line 258 in _run_main File "/home/kp1/google-research/fad-env/lib/python3.7/site-packages/absl/app.py", line 312 in run File "/home/kp1/google-research/frechet_audio_distance/create_embeddings_main.py", line 75 in File "/usr/local/lib/python3.7/runpy.py", line 85 in _run_code File "/usr/local/lib/python3.7/runpy.py", line 193 in _run_module_as_main Segmentation fault (core dumped)

Darn Im also starting to experience the same issue now, here's my logs:

2021-08-05 22:49:02.936735: F tensorflow/stream_executor/cuda/cuda_dnn.cc:533] Check failed: cudnnSetTensorNdDescriptor(handle_.get(), elem_type, nd, dims.data(), strides.data()) == CUDNN_STATUS_SUCCESS (9 vs. 0)batch_descriptor: {count: 5979 feature_map_count: 64 spatial: 96 64 value_min: 0.000000 value_max: 0.000000 layout: BatchDepthYX} Fatal Python error: Aborted

Thread 0x00007ff65048b700 (most recent call first): File "/opt/conda/lib/python3.8/threading.py", line 870 in run File "/opt/conda/lib/python3.8/threading.py", line 932 in _bootstrap_inner File "/opt/conda/lib/python3.8/threading.py", line 890 in _bootstrap

Thread 0x00007ff64dc8a700 (most recent call first): File "/opt/conda/lib/python3.8/threading.py", line 306 in wait File "/opt/conda/lib/python3.8/threading.py", line 558 in wait File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/worker/data_plane.py", line 207 in run File "/opt/conda/lib/python3.8/threading.py", line 932 in _bootstrap_inner File "/opt/conda/lib/python3.8/threading.py", line 890 in _bootstrap

Thread 0x00007ff674084740 (most recent call first): File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1441 in _call_tf_sessionrun File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1349 in _run_fn File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1365 in _do_call File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1358 in _do_run File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1180 in _run File "/opt/conda/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 957 in run File "/home/jovyan/scratch-01/rs002/tt-vae-gan/fad/frechet_audio_distance/audioset_model.py", line 69 in process_batch File "/home/jovyan/scratch-01/rs002/tt-vae-gan/fad/frechet_audio_distance/create_embeddings_beam.py", line 247 in _flush_buffer File "/home/jovyan/scratch-01/rs002/tt-vae-gan/fad/frechet_audio_distance/create_embeddings_beam.py", line 283 in finish_bundle File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/worker/bundle_processor.py", line 999 in process_bundle File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 639 in process_bundle File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/worker/sdk_worker.py", line 601 in do_instruction File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/portability/fn_api_runner/worker_handlers.py", line 378 in push File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 905 in process_bundle File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 603 in _run_bundle File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 557 in _run_stage File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 366 in run_stages File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 202 in run_via_runner_api File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 191 in run_pipeline File "/opt/conda/lib/python3.8/site-packages/apache_beam/runners/direct/direct_runner.py", line 131 in run_pipeline File "/opt/conda/lib/python3.8/site-packages/apache_beam/pipeline.py", line 564 in run File "/home/jovyan/scratch-01/rs002/tt-vae-gan/fad/frechet_audio_distance/create_embeddings_main.py", line 70 in main File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 251 in _run_main File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 303 in run File "/home/jovyan/scratch-01/rs002/tt-vae-gan/fad/frechet_audio_distance/create_embeddings_main.py", line 75 in File "/opt/conda/lib/python3.8/runpy.py", line 87 in _run_code File "/opt/conda/lib/python3.8/runpy.py", line 194 in _run_module_as_main

[gudgud96]

You can find mel_features.py in tensorflow_models/audioset/vggish/. To fix this and oother import problems just create tensorflow_models/audioset/__init__.py and add the line: from .vggish import mel_features, vggish_params, vggish_slim Also, in file tensorflow_models/audioset/vggish/vggish_slim.py you should change the import to from . import vggish_params as params.

You will also have problems with Tensorflow version: for me it worked r1.15 and changing in file frechet_audio_distance/fad_utils.py line 37 to tf_record = tf.python_io.tf_record_iterator(filename).__next__()

An update here on recent dig-in on FAD - TF 2.+ should work, I am using the more recent TF 2.11.0 but only changing line 37 as suggested by Javier, it works fine on my side. The import related changes by Javier are still needed though.

Thanks again @javiernistal for this comment, it's golden!

[FelipeMarra]

So, to solve the Clone TensorFlow models repo into a 'models' directory step:

mkdir tensorflow_models
touch tensorflow_models/__init__.py
git clone https://github.com/tensorflow/models.git
mv models/research/audioset tensorflow_models/audioset/
rm -rf models/
touch tensorflow_models/audioset/__init__.py
vi tensorflow_models/audioset/__init__.py
# add the line:
# from .vggish import mel_features, vggish_params, vggish_slim
vi tensorflow_models/audioset/vggish/vggish_slim.py
# change the import to:
# from . import vggish_params as params.
pip install --upgrade tf_slim
vi frechet_audio_distance/fad_utils.py
# change line 37 to:
# tf_record = tf.python_io.tf_record_iterator(filename).__next__()
vi frechet_audio_distance/audioset_model.py
# change line 90 to:
# for i in range(0, samples - vggish_params.SAMPLE_RATE + 1,

It worked using python 3.10.12 and tensorflow 2.17.0. The current commit of tensorflow/models is this one.

Edit: Looks like people from AudioCraft also documented how to run FAD here.