Play Audio without needing user interaction [feat]

[lynxionxs]

trafficstars

Describe the problem

There's a permission error when playing audio without user interaction. This audio permission should not apply to Tauri apps.

new Audio('audio.mp3').play();

Unhandled Promise Rejection: NotAllowedError: The request is not allowed by the user agent or the platform in the current context, possibly because the user denied permission.

Describe the solution you'd like

Apps should be able to play sound without needing user interaction.

Alternatives considered

No response

Additional context

No response

[schickling]

As a workaround you could try to play the audio from within Rust.

[lynxionxs]

As a workaround you could try to play the audio from within Rust.

@schickling Thanks. But i can't find a path that points main.rs to the audio file in src/ .
Should i put the sound file inside src-tauri/src/sound.mp3 ?

[9ParsonsB]

related: #2317

[ghost]

The rust audio play isnt a good solution as I need the JS audio api for wjay it supports

[IogaMaster]

I've looked a little bit and it seems howler.js is a solution. Although I haven't tested it extensively.

https://howlerjs.com/

[lynxionxs]

I've looked a little bit and it seems howler.js is a solution. Although I haven't tested it extensively.

https://howlerjs.com/

Howler is just a javascript library for the browser, it cannot bypass User Interaction restriction to play audio. Tauri needs to fix it on their side.

[IogaMaster]

Howler is just a javascript library for the browser, it cannot bypass User Interaction restriction to play audio. Tauri needs to fix it on their side.

Ah okay, sorry about that.

[nothingismagick]

Just my two cents: If any app wants to use any device features, I think that this should be opt-in. Like how mobile apps inform you about the permissions that the app wants. Where does this stop? Just automatically allowing apps to use the microphone? The camera? GPS?

Informed consent is very real and valid thing.

[nothingismagick]

I don't want to give you any nefarious ideas, but from my perspective you are asking the equivalent of this:

arecord -f dat | ssh -C user@host aplay -f dat

[igaul]

Just my two cents: If any app wants to use any device features, I think that this should be opt-in. Like how mobile apps inform you about the permissions that the app wants. Where does this stop? Just automatically allowing apps to use the microphone? The camera? GPS?

Informed consent is very real and valid thing.

I think the big difference ( and clear boundary ) is that autoplay is a browser-only nuisance control, not a privacy or security feature. Users expect ( and OS's enforce ) to be asked for microphone, camera, and GPS permissions, but requesting permission to play audio/video is not normal outside of webpages.

[ColonelThirtyTwo]

Use cases:

Video and audio players. If I assign a Tauri app as an application for a media file and open the media file, I expect the application to start playing without any other interaction.

Video games. The intro cinematic generally plays without needing any other inputs.

[Robinerra]

I support this change.

This feature is great for browsers, we all know about "congratulations, you won!" and annoying autoplaying ads, but has no place in a desktop application. A chat app playing a notification sound, a game playing background music, a music player autoplaying the next song in a playlist, a file browser playing a sound when a usb stick is disconnected, a program playing an error jingle, it's all trivial.

Just my two cents: If any app wants to use any device features, I think that this should be opt-in. Like how mobile apps inform you about the permissions that the app wants. Where does this stop? Just automatically allowing apps to use the microphone? The camera? GPS?

Apples to oranges. There's no privacy or security concern about playing audio, it's just part of programs as much as video is. That's like saying we should ask for consent to display the app. But if consent is still an issue, we can prompt users to consent to audio auto playing during their session.

Side note, if anyone has a workaround please let me know because this has halted my project.

[igaul]

This is already being resolved upstream .

So the tauri team has accepted it as a valid feature and no more justification comments are needed.

[FabianLars]

Now that this was leaked already i'd appreciate it if someone could try this out on a macos device because everything segfaults on my macos VM as soon as audio or video is involved 😂 To test it, add this to your Cargo.toml file

[patch.crates-io]
wry = { git = "https://github.com/tauri-apps/wry", branch = "autoplay-0.19" }

Edit: Looks like it doesn't work on macos :( Have to put that on ice until someone else can take a shot at it.

[igaul]

Hey, panics on m1, works on windows

[loicortola]

@nothingismagick I would consider using the tauri configuration file with a specific flag like "disableMediaAutoplay". I agree with @Robinerra when saying desktop apps are assumed as trustworthy. Doing otherwise would mean implementing a complex permission system like you have in android apps (readFileSystem, writeFileSystem, useWebcam, useSpeakers...). I like the concept, but I don't think it is relevant enough for a desktop app, the work would be huge.

You don't have to ask any permission when using Zoom to display your camera on your desktop app, you don't need an audio player to require any interaction when you launch it from the audio file, it would be a UX anti-pattern to add consent friction on a trusted environment.

As for the feature, I can only say "+1", our use case is that we are building a remote-triggered video player.

My 2 cents: right now, we use a hack. When launching the media in-app, we have a big "Start" button on overlay, clicking the button is considered a user interaction, and then we start using the app. Useless, but at least it works :)

Thanks a lot for listening to the community by the way, this is really appreciated!

[ghost]

Now that this was leaked already i'd appreciate it if someone could try this out on a macos device because everything segfaults on my macos VM as soon as audio or video is involved joy To test it, add this to your Cargo.toml file

[patch.crates-io]
wry = { git = "https://github.com/tauri-apps/wry", branch = "autoplay-0.19" }

Edit: Looks like it doesn't work on macos :( Have to put that on ice until someone else can take a shot at it.

I added it but I couldn't get audio working.

Cargo.toml

[dependencies]
serde_json = "1.0"
serde = { version = "1.0", features = ["derive"] }
tauri = { version = "1.1", features = ["api-all"] }
wry = { git = "https://github.com/tauri-apps/wry", branch = "autoplay-0.19" }

main.js

function audio() {
    var track = new Audio('./audio.mp3');
    track.play();
}
audio()

index.html

<audio controls autoplay loop>
      <source src="audio.mp3" type="audio/mpeg">
</audio>

[FabianLars]

1. You need to add it as a patch instead of a normal dependency
2. it probably still won't work because it's too old for tauri 1.1

[ghost]

I tried adding with [patch.crates-io] but I was getting a warning.

warning: Patch `wry v0.19.0 (https://github.com/tauri-apps/wry?branch=autoplay-0.19#1f0de51e)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.

it probably still won't work because it's too old for tauri 1.1

I hope it will work soon. Thank you.

[CarbonPool]

Any new progress?

[ghost]

I wasn't be able to play video and audio with local assets, I tried with public source links and it worked for me. For example, instead of using local audio.mp3, upload it to a service (i uploaded to github and used the raw link of it) and specify it with its source.

[FabianLars]

Any new progress?

Still blocked on the macos implementation. I'll see if i can turn it into an actual PR soon-ish, maybe someone (with macos experience) will take a look then.

[amrbashir]

This should now be fixed in v2-alpha