Integrating Pitch Shifting into a Custom HTML5 Video Player with Tone.js

[rrehltp]

I’m trying to integrate Tone.PitchShift with an HTML <video> element in my project, but the pitch shifting functionality isn’t working as expected. Here’s a summary of my setup and the issue:

Setup:

• Tone.js Version: v14.7.39
• Browser: Safari
• Code Snippet:

constructor() {
    this.player = new CustomPlayer();
}

setupPitchShift = () => {
    this.pitchShift = new Tone.PitchShift().toDestination();
    const mediaSource = Tone.context.createMediaElementSource(
      this.player.container
    );
    Tone.connect(mediaSource, this.pitchShift);
};

Issue:

• The video plays, but the pitch shifting does not apply to the audio.

Steps to Reproduce:

1. Initialize a Tone.PitchShift node.
2. Connect an HTML <video> element as the audio source.
3. Attempt to modify the pitch using pitchShift.pitch.

Questions:

1. Is there a specific configuration or connection order required for MediaElementSource and PitchShift?
2. Are there any known limitations with using PitchShift on <video> elements?

Any guidance would be greatly appreciated.

Best regards

[tambien]

Are you initiaing the Web Audio context with Tone.start() on a click event?

[rrehltp]

@tambien I started Tone.js with Tone.start() with a click event, and confirmed the state of Tone context is running. But Still pitchShift not working.

Here, this.player.container is HTML

Below is how I changed pitch with slider.

this.pitchShift.pitch = parseInt(e.target.value);

[thamilton-vs]

@rrehltp It is not issue in Tone.js. Safari has notable limitations with audio processing, especially around Web Audio API features like AudioWorklet, and Tone.js doesn’t work as expected.