jmshrv
redesign-beta: Volume Normalization short delay noticable
The moment a song is finished and a new one begins finamp takes around 0.1~ seconds to adapt to the new normalization dB. This is especially noticable if the following song has higher volume than the first one (eg from -10dB to -4dB) because then the first 0.1-0.2 seconds are very loud, thus "ruining" gapless play.
Sorry if I mixxed things up with the dB values. I am always confused about which value means a lower volume, and which one is louder.
Is this on Android or iOS?
I can try to change when/how the new gain is applied, but it will probably be hard to time it exactly. Maybe a gradual fade over a short duration would work better 🤔
Android. v0.9.7. I believe a short-duration fade is a good idea. However, the fade should complete just before the next track starts to avoid any noticeable transitions.
Is the volume normalization data for the next track already pre-buffered? If no further calculation is required, simply setting the new volume at the moment the next track begins should solve the issue.
simply setting the new volume at the moment the next track begins should solve the issue
Well, in a way that's exactly what we're currently doing. We wait for an event that tells us that a different track is playing. It seems like the event might be slightly delayed in some cases, so maybe we can improve the timing somehow.
Which phone are you using?
I noticed the same issue on my Pixel 6a. But only for tracks with vastly different normalization. In my case going from a +5dB song to a -4dB song that starts right at the beginning of the file.
I can reproduce this on Pixel 7 with 0.9.12 It is very annoying and distracting.
Is the delay always the same? Or does it depend on what you're playing or maybe how big the queue is?
When testing out the limits of the queueing system (queueing up thousands of tracks), I noticed that the delay was becoming greater, but that could've been a general slowdown of the app.
My playlist is a sort of "living playlist" and hovers around ~130 songs. I only really notice the delay in normalization if the song I'm currently listening to has a really high normalization value (the unmodified file is quiet) and the next song in the playlist has a small normalization value AND the music starts immediately at the beginning of the file. I'd say the delay is less than 250ms.
Is the delay always the same? Or does it depend on what you're playing or maybe how big the queue is? When testing out the limits of the queueing system (queueing up thousands of tracks), I noticed that the delay was becoming greater, but that could've been a general slowdown of the app.
My playlist is ~200 songs. It's only really noticeable if the preceding song is much quieter. Maybe lasts 0.5s at the most, maybe more like 300-400 ms. The affect is very short but quite shocking. I'm mostly restating what @solidsnake1298 said haha
@Chaphasilor asked me in #1302
Now that you've dealt with the code a bit more closely, do you have any new ideas how we could mitigate https://github.com/jmshrv/finamp/issues/769? I'm guessing the code is too unrelated to actual playback though :)
Actually it is related. I didn't understand it from the start of this text, but now I am completing it rewriting this sentence. I think the fact that this volume normalization is working at all is a bug in just_audio or Finamp itself (there is a condition that Finamp applies or does not apply a time correction to UI timeline, idk it's 2 AM). Sounds crazy, but let me explain. Also consider the fact that my knowledge may be outdated and i didn't test this hypothesis. Also any mention of headphones mentions bluetooth headphones.
Almost any digital playback hardware connected to general-purpose PC (including a mobile phone) has a buffer. This buffer is a measure to the CPU not being fast enough to feed the data (imagine a 44100 Hz scheduler and compare to Linux scheduler being currently at 100 Hz), and this results to "delay" between audio signal source and actual playback. In most cases it is small enough, however not tolerated by some software (~~e.g. osu! rhythm game, I get 5ms latency there including my own dtimulus-response chain, while I expect the buffer to be somewhere near 10 ms..~~ no, the buffer 10 ms leads to expected latency 5 ms (that was on Windows though), most probably osu didn't predict that but it could be that they considered that).
For example, let's see my latency (that's pw-top - I didn't even know about it, not mentioning it installed on my PC without my knowledge):
1024 quants at 48 kHz sampling rate. 1024/48000 = 0.021(3) seconds buffer (I again changed my mind about osu! fixing that ha-ha, need to redo the experiment on linux).
THE INTRODUCTION TO AUDIO BUFFERING ENDS HERE
But 21 ms is pretty much small to be sensed by a human. Except on android, I think, it is larger, due to diverse CPUs. But there is even larger buffer: bluetooth. With new audio fade out and in it is very noticeable: you press pause, wait and then hear the fade out. That's a great example of a latency bug: both volume and playback pause/resume are controllable outside of audio stream (e.g. volume buttons are responded instantly) and so fading in/out uses something wrong.
The bluetooth buffer can go as far as 0.5s (or more? idk). osu!, for example, does not mitigate such big latency (you can cut 20 ms from the start of hitsound but not 0.5s), but youtube does - and the speech is perfectly synchronised with the picture (the video is delayed by the same value as buffer size).
And just_audio should mitigate that too. As it does not mitigate that (verification needed), Finamp doesn't know the exact time, and so can't set the volume in exactly the gap between tracks. But the fact that just_audio does volume change inside the audio stream allows the Finamp to synchronise the volume with the playback. Otherwise the delay would be negative, and the gain would be changed before the track is played.
The fact that it changes the volume inside the audio stream has its own drawbacks, namely: inability to change the volume of already sent frames. As finamp doesn't change gain before the frame starts (it is either the frame from singular track itself or concatenated by ConcatenatingAudioSource - anyway sent before async code has any chance to run), DAC gets the frames without changed volume. This, I think, is the reason of that issue. Async is not exact, async is always late, async is async. If you can schedule it at exact time - then it does not matter, but finamp doesn't do that:
https://github.com/jmshrv/finamp/blob/fc69dd53ab7220528c0c44036842227eeed5ea9d/lib/services/music_player_background_task.dart#L300-L347
It triggers on the moment when tracks are changed. When the event is fired, it is already too late. And the code is async, it doesn't block anything.
So, we have two bugs overlapped, hiding the true cause but not completely.
Also some text that I wrote and then forgot why
Finamp UI is synchronised with playback with headphones, but it is slightly faster than playback without headphones (slightly! We need very experienced listeners to actually confirm that, or simply 8000 Hz sample vs 16000 Hz sample and visualise in any software). To see that, I used gapless tracks from different albums (so that there's no gap but the track change is audible).But in both cases the delay of replaygain is positive, and most probably slightly bigger on headphones (but not as big as delay of "pause" button, so minus one hypothesis). I had something something in my head due to that but I forgot.
Anyway, there is two ways to fix this issue:
- exact time when next track will play should be known ahead of time. This means knowing exact latency and exact time when just_player sent the data to DAC. This will allow to schedule a task at exact time (+- the resolution of scheduler) and this task will set the volume avoiding the audio stream. For maximum performance (I repeatedly said in my PR - performance performance, I didn't know it is already too late) the value should be precalculated, but it anyway allows to change it on the fly.
- Also, the gain can be baked into the audio stream as well (actually the first thought then I faced this issue myself, back in february I think). This will require remuxing the stream if setting is changed (so no changes on the fly), but otherwise that's a very cheap fix.
Now I'll go to sleep. I hope this is not nonsense.
@HeroBrine1st thanks for the detailed explanation! This of course makes a lot of sense, mainly the fact that Finamp applies the gain in a reactive way after the track has already changed.
So what you're saying is we should instead "schedule" (e.g. with a high-precision timer?) the exact moment when the track is about to change, and there apply the gain (which we calculated previously).
I think that sounds like a reasonable solution, if there is such a way to reliably schedule a function execution.
We'll have to make sure to re-schedule this whenever playback is paused, seeked, or buffering of course, but that should be doable.