spleeter

spleeter copied to clipboard

Description

This looks awesome, is there a way to use it as a node module? (eg via npm) packaged in such a away to be able to use it in an electron app?

Additional information

Thinking of adding it as an option to autoEdit.io to clean audio before sending it to a Speech To Text Enginge.

Hi @pietrop

There's no out of the box node module made on top of spleeter that I know of.

Theoretically it's feasible to build a javascript lib out of spleeter, you'd need to use the tensorflow.js port to load the models though.

An easier possibility would be to host a small webservice serving spleeter and requesting it through js I guess.

Theoretically it's feasible to build a javascript lib out of spleeter

Could you kindly elaborate more? does this mean porting spleeter python code to js?

We are about to release a js framework with all the required pre and postprocessing soon. Stay tuned

@faroit I’ve just started working on a Spleeter “port” on top of TFJS for consumption in electron apps. Would love to contribute however I can - let me know if there’s interest in external contributions pre release!

Thanks for your great work on this! 🙏 I'm very interested in a TFJS version as well and am looking forward to this upcoming release!!!

@faroit any news?

@chenei we are updating the framework to support tf 2.2. ETA next two weeks. @TheoBr we could could for sure get some help. We only implemented the pre- and postprocessing. So the spleeter spectrogram core model would need to be converted to tfjs (as we add open-unmix for now). The last time I tried it was not compatible out of the box and required manual layer patching.

@TheoBr Hi, Can I help u somehow?

@chenei I have not gotten started on any work related to Spleeter in Node :( Would love an update or even a WIP branch to work against if at all possible @faroit

@faroit any news?

@AdrianButnar @TheoBr sorry for the slow response. We managed to get the full open-unmix and spleeter models working. If any of you are js devs, we would love some feedback. Send me a mail please

@faroit thanks for your past update and work on TF.js version of spleeter! is there a timeline for when code and/or artifacts will be released?

@faroit is this still a thing? Has support been dropped?

@theblissprogrammer @Syps @AdrianButnar @TheoBr sorry for the long delay. We have working models but we are currently working on speeding up inference and reduce memory usage.

See a W.I.P site here (currently only open-unmix works) https://sigsep.github.io/open-unmix-js/

We will open the repo very soon and can't wait to get some help from javascript gurus ;-)

Just to let you know, I just tried a Open-Unmix vocal separation & the processing bar circle just spun for several minutes with no results.

Roger

 On Thursday, November 12, 2020, 08:05:23 AM CST, Fabian-Robert Stöter <[email protected]> wrote:  

@theblissprogrammer @Syps @AdrianButnar @TheoBr sorry for the long delay. We have working models but we are currently working on speeding up inference and reduce memory usage.

See a W.I.P site here (currently only open-unmix works) https://sigsep.github.io/open-unmix-js/

We will open the repo very soon and can't wait to get some help from javascript gurus ;-)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Just to let you know, I just tried a Open-Unmix vocal separation & the processing bar circle just spun for several minutes with no results. Roger On Thursday, November 12, 2020, 08:05:23 AM CST, Fabian-Robert Stöter

same thing happened with the file i tried. it was a 4 minute mp3 and i let it spin for more than 20 minutes with no result.

i figured i'd give feedback as well (just in case it helps).

good luck with the speed/memory improvements and thanks for all your work @faroit!

@Mixerrog @skratchdot yes. RAM is an issue in long files. For now:

• use chrome
• try <30s files

Please wait until we open the repo for reports as this is out of scope for spleeter. ( Or just drop me a mail)

Thanks you for this amazing project. Look forward for JS version :)

I've been able to use spleeter it in my Node.js application with Python Shell along with node-wget.

@amamenko Could you elaborate pls with a snippet for the command line you're passing? It's about an open source project I'm working on

@kyr0 Sure, so I have a couple of Python scripts in separate files that are both executed by the Node.js python-shell module.

The first file is called "install_package.py," which takes an argument and downloads Python packages accordingly:

import os
import sys
 
os.system('python3 -m pip install {}'.format(sys.argv[1]))

The second file is called "spleeter_stems.py," which takes an argument and runs Spleeter on that argument file via the command line:

import os
import sys

os.system('spleeter separate -p spleeter:2stems -o output {} -c mp3'.format(sys.argv[1]))

A basic Node.js example using python-shell and node-wget with these two Python scripts looks something like this:

const fs = require("fs");
const wget = require("node-wget");
const { PythonShell } = require("python-shell");

wget("https://github.com/deezer/spleeter/raw/master/audio_example.mp3", () => {
  const filePath = "audio_example.mp3";

  // Make sure Spleeter is installed
  PythonShell.run(
    "./python_scripts/install_package.py",
    { args: ["spleeter"] },
    (err) => {
      if (err) {
        throw err;
      } else {
        console.log("Splitting audio file.");
        // Get audio file
        const spleeterOptions = {
          args: ["audio_example.mp3"],
        };

        // Split audio into stems and clean up
        PythonShell.run(
          "./python_scripts/spleeter_stems.py",
          spleeterOptions,
          (err) => {
            if (err) {
              throw err;
            } else {
              console.log("Successfully split track into two stems");
              fs.unlinkSync(filePath);
              fs.rmSync("pretrained_models", { recursive: true });
              console.log(
                "Removed pretrained_models directory and local audio file"
              );
            }
          }
        );
      }
    }
  );
});

@amamenko Wow, nice, thank you Avi! This is perfect, I'll play with it this weekend and see if I can quickly come up with a general purpose UI solution for all OS

@amamenko i used your code and it's working very good thanks

Hey @faroit @TheoBr, just checking on updates for the tf.js port or any JS version released for the spleeter module.

Is there anything along the lines in progress or in backlog? Thanks!

Hey @faroit @TheoBr, just checking on updates for the tf.js port or any JS version released for the spleeter module.

Is there anything along the lines in progress or in backlog? Thanks!

No update on that. We released the umxjs code and all four stem models.

@kyr0 Sure, so I have a couple of Python scripts in separate files that are both executed by the Node.js python-shell module.

The first file is called "install_package.py," which takes an argument and downloads Python packages accordingly:

import os
import sys
 
os.system('python3 -m pip install {}'.format(sys.argv[1]))

The second file is called "spleeter_stems.py," which takes an argument and runs Spleeter on that argument file via the command line:

import os
import sys

os.system('spleeter separate -p spleeter:2stems -o output {} -c mp3'.format(sys.argv[1]))

A basic Node.js example using python-shell and node-wget with these two Python scripts looks something like this:

const fs = require("fs");
const wget = require("node-wget");
const { PythonShell } = require("python-shell");

wget("https://github.com/deezer/spleeter/raw/master/audio_example.mp3", () => {
  const filePath = "audio_example.mp3";

  // Make sure Spleeter is installed
  PythonShell.run(
    "./python_scripts/install_package.py",
    { args: ["spleeter"] },
    (err) => {
      if (err) {
        throw err;
      } else {
        console.log("Splitting audio file.");
        // Get audio file
        const spleeterOptions = {
          args: ["audio_example.mp3"],
        };

        // Split audio into stems and clean up
        PythonShell.run(
          "./python_scripts/spleeter_stems.py",
          spleeterOptions,
          (err) => {
            if (err) {
              throw err;
            } else {
              console.log("Successfully split track into two stems");
              fs.unlinkSync(filePath);
              fs.rmSync("pretrained_models", { recursive: true });
              console.log(
                "Removed pretrained_models directory and local audio file"
              );
            }
          }
        );
      }
    }
  );
});

Did anyone manage to get this implementation working? Getting errors about the pretrained_models missing. But even before that the result from running the PythonShell is null.