ruffle
ruffle copied to clipboard
ruffle-rs
[regression] weird sound with Firefox extension in Windows (full screen only)
Describe the bug
After #4273 was merged, the Firefox extension plays choppy sound with the animation at https://www.free-culture.tk/.
It only happens in Windows in fullscreen mode. I cannot reproduce it in Linux (either fullscreen or not).
I wonder whether #7526 might fix this (in any case, many thanks for the patch, @torokati44).
Expected behavior
Sound playing fine, no matter whether fullscreen mode is used or not.
Affected platform
Browser's extension
Operating system
Windows
Browser
Firefox 103
I am also getting choppy audio in Firefox now. It's extremely noticeable in Prizma Puzzle Challenges - the main menu music sounds terrible. Everything is perfectly fine in earlier Ruffle builds and in Chrome / Chromium browsers. What's interesting to me is that I've always had this exact same issue with WAFlash, so maybe Firefox is just bad at running audio decoding inside of WebAssembly, or something like that?
Nevermind, it was a problem with my Firefox settings. Somehow I did not have "Use recommended performance settings" turned on:
Now that I have switched it on, the audio is nearly 100% fine (maybe slightly more choppy than it used to be, but nothing noticeable unless I really pay close attention).
@torokati44, I guess this may be related to non-standard sample rates (such as #6673).
Although tagged as MP3 11Khz 16Bit mono in the SWF file, the sample rate is 16kHz.
Is this what may be causing the choppy audio?
@n0samu, if I could ask you for a favor, could you check #7533 without hardware acceleration?
@Herschel,
I have the impression that this issue is related to #7533.
It seems that displaying the full-screen image in a 27″-monitor eats up so much resources (using only software, no hardware acceleration), that there is not much left for symphonia to decode the MP3 sound.
I think this is why choppy sound only happens when the presentation is in full-screen mode.
And it doesn’t happen on my Linux computer (even if I disable hardware acceleration), because my laptop has only a 17″-monitor.
Just in case it might help. Many thanks for your help.
Having said that, it would be great to have the calculation of Sound.duration() with non-standard sample rates fixed too (#6673).
Many thanks again for your help and your excellent work.
@n0samu,
I have updated the selfhosted extension at https://www.free-culture.tk to nightly from 2022-08-10.
Would you be so kind to test if sound is played perfectly fine (with hardware acceleration disabled) with Chrome?
Many thanks for your help again.
Sound is played fine on a Windows computer with a 27″-monitor using Chrome at full screen.
This is the same computer in which Firefox cannot play without distortion (kind of stuttering, apparently due to lack of resources) the same presentation in full screen.
Just in case it may help to fix the issue.
@Herschel,
I don’t know whether this #7669 intended to fix this.
With a larger buffer, sound stutters with longer chunks.
I mean, audio is played a bit longer, but it is paused before it is played again.
Just in case it helps and many thanks for your help.
#7533 was generated and fixed by Firefox.
But the weird sound playing probably comes too much resources dedicated to the image rendering (with no hardware acceleration and the use of symphonia as audio decoding library (which seems to need more resources).
I wonder whether any improvements from @Aaron1011 related to Bitmap may help here.
Since this issue seems to be related to Firefox not having enough resources to play sound with Symphonia, I wonder whether #7909 could be adapted to play sound when Symphonia cannot play it flawlessly.
As far as I understand, audio playback is going to remain choppy unless either we go back to native audio decoding (unlikely) or we implement audio decoding on a separate thread via AudioWorklet. In the meantime, I think it would be helpful if there was a way to choose whether to decode audio natively or via Rust on the web - the Rust-based decoding is more reliable, but it suffers from this performance issue.
@n0samu,
many thanks for your clear explanation.
Until sound playing would be implemented using AudioWorklet, I wonder whether there could be a configuration option in JS such as audio-decode with possible values native or rust (being rust the default option).
Would it be doable in a reasonable time frame?
Many thanks for your help.
Would it be doable in a reasonable time frame?
I wish I could say yes but honestly I have no idea :(
@n0samu, many thanks for your honest reply.
Since @Herschel self-assigned this issue, I wonder whether he may be interested in my proposal before implementing the AudioWorklet (which I guess, it might take longer [and other code is more important (such as #5492)]).
Today I discovered something weird.
https://www.free-culture.tk/lfc-presentation.swf has choppy audio playing when in full screen.
https://www.free-culture.tk/free_culture.swf contains basically the same audio file and audio playing at full screen is fine.
In both cases, Firefox has hardware acceleration disabled.
Any idea about what might be making the difference in audio playing quality?
I‘m afraid I don’t have any access to the computer which caused with the original issue.
The newer computer that replaced it displayed the presentation perfectly fine.
Since I seem to be the only user having experienced the issue, I’m closing it myself.