Questions about pre-trained model

[sophia1488]

Hi, I'm really interested in your work and would like to test your model on some pop songs! But I failed to download the pre-trained model. Could you put your checkpoint at Google Drive or some other available platforms? 🥺

Also, I'm pretty curious about your model size, (I didn't find it in your recent paper), given that TFC-TDF-U-Net v1 is fairly small.

Look forward to your reply!

[ws-choi]

Hi @sophia1488,

We had used git-lfs to distribute the pretrained models, but we found that it goes unavailable frequently due to github's limited bandwidth unless we purchase an additional package.

We are currently reorganising code to distribute it via zenodo.

I will update this repo as soon as possible if it is finished.

sorry for the inconvenience.

[0xBEEEF]

I'm sorry that I now have to follow up here as well. But unfortunately I am also not able to test the experiment, exactly because of the errors mentioned. Since now several months have passed, I would be very connected that there is any solution here, how to run the repository.

With me it fails now for weeks to clone the thing. So either I always catch the wrong time, or it is tried by so many (which is great) again and again.

So my request to @ws-choi : would it be possible as suggested above to make the model available to the public in the short term via some other source? I think this is all great, and would test the work done here.

Thanks in advance!

[ws-choi]

Hello everyone,

I pushed an updated branch 'leaderboard_A'.

• Updated

  • branch leaderboard_A

    • The updated branch 'leaderboard_A', hopefully, works in any environment including Windows.
    • You can download checkpoints from Zenodo instead of git-lfs. Please read this README.md
    • Please leave some comments or questions if you have, especially if you encounter errors.

  • branch leaderboard_B

    • I will update the other branch until April 30th.

I am deeply sorry for my late response.

I was refactoring this repo for better readability and more reliable checkpoints downloading last year but I have been busy for other projects. Currently I am writing a paper and the deadline is too soon (April 10th). I will update it as soon as possible. Sorry.

Best, Woosung

[ws-choi]

One more thing! I tested it on my Windows-laptop and it worked fine.

If one of you could successfully run it with the updated branch on Windows then please let us know! Then we will change the line below in current README.md:

The repository supports Linux and https://github.com/kuielab/mdx-net-submission/issues/1 (MacOS untested).

[Zokhoi]

@ws-choi

You can download checkpoints from Zenodo instead of git-lfs.

The test separation song Mu - Too Bright is still on git-lfs, so that may still throw an error.

One more thing! I tested it on my Windows-laptop and it worked fine.

If one of you could successfully run it with the updated branch on Windows then please let us know!

I tested it on windows and it ran fine, however the command line instructions may need to be modified a bit for windows. I can update the documentation for that.

[ws-choi]

Hi @Zokhoi , thank you for reporting this issue. I removed all the git-lfs sample waves and directly pushed a wav file to leaderboard A branch.

The mixture track used is La Vie En Rose (FR) by Stella Jang

[Zokhoi]

The mixture track used is La Vie En Rose (FR) by Stella Jang

@ws-choi This track only has vocals and acoustic guitar, meaning only the "other" and "vocals" stems extracted would be meaningful. Maybe you can select another track also with drums and bass to better demonstrate the models?

[ws-choi]

I got your point, but currently it's the only track that I can upload due to the copy-write issue. After I discuss it with other artists, I will upload other tracks :)

[sophia1488]

Hi, thanks for putting great efforts into this! And good luck with your paper~

I followed the instructions on Readme - installation for Linux environment, and it was fine.

But when I tried to run separation (python predict_blend.py), the following error occurred. 😥

onnx_A/bass.onnxTraceback (most recent call last):
  File "~/.local/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 280, in __init__
    self._create_inference_session(providers, provider_options)
  File "~/.local/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 307, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
RuntimeError: /onnxruntime_src/onnxruntime/core/platform/posix/env.cc:142 onnxruntime::{anonymous}::PosixThread::PosixThread(const char*, int, unsigned int (*)(int, Eigen::ThreadPoolInterface*), Eigen::ThreadPoolInterface*, const onnxruntime::ThreadOptions&) pthread_setaffinity_np failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/mdx-net-submission/evaluator/music_demixing.py", line 147, in run
    self.evaluation()
  File "~/mdx-net-submission/evaluator/music_demixing.py", line 134, in evaluation
    self.prediction(mixture_file_path=self.get_music_file_location(music_name),
  File "~/mdx-net-submission/predict_blend.py", line 26, in prediction
    sources = self.demix(mixture_file_path)
  File "~/mdx-net-submission/predict_blend.py", line 33, in demix
    base_out = self.demix_base(mix)
  File "~/mdx-net-submission/predict_blend.py", line 60, in demix_base
    _ort = ort.InferenceSession(f'{onnx_name}/{model.target_name}.onnx')
  File "~/.local/lib/python3.9/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 283, in __init__
    print("EP Error using {}".format(self._providers))
AttributeError: 'InferenceSession' object has no attribute '_providers'

[ws-choi]

Hi @sophia1488, I am not for sure but I think it's a conflicting library dependency issue. Did you install onnxruntime-gpu instead of onnxruntime ?

[sophia1488]

Hello, I installed onnxruntime, same as environment.yml.

$ pip list | grep onnxruntime
onnxruntime             1.7.0

[ws-choi]

I am sorry but I have no idea what causes the error for your case.
This is a Colab-version example I just have made. I hope it helps.

[Zokhoi]

@sophia1488 Reference microsoft/onnxruntime#8313

Maybe set this:

options = ort.SessionOptions()
options.intra_op_num_threads = 1
options.inter_op_num_threads = 1

before predict_blend.py line 59?

[yzhou61]

@ws-choi thanks so much for your work! Would you be able to provide the onnx models for download for leaderboard B in the mean time?

[sophia1488]

Thank you guys, it's working! :)

[ws-choi]

Hi @yzhou61 and @sophia1488 , I will provide onnx models for leaderboard B by at the end of April. It might take more time though because I am preparing to relocate these days, sorry,

[ws-choi]

Hi @yzhou61 , @sophia1488 , I had a network connection issue for few days, and finally uploaded it today. I pushed a new branch for leaderboard B : https://github.com/kuielab/mdx-net-submission/tree/lbB_zenodo. It works find in my side. I did not merge it to the original branch, please tell me if you have any problems.

[euniwang333]

Hi @yzhou61 , @sophia1488 , I had a network connection issue for few days, and finally uploaded it today. I pushed a new branch for leaderboard B : https://github.com/kuielab/mdx-net-submission/tree/lbB_zenodo. It works find in my side. I did not merge it to the original branch, please tell me if you have any problems.

Hi @ws-choi , Thanks for the great work for this project! I can inference the .onnx file and get good performance. But I'm wondering if it can be re-trained from your pretrained model. Is the .ckpt file is required if need to do re-trained? If yes, can you provide .ckpt file, too?

Thanks so much