Auxio
Auxio copied to clipboard
OxygenCobalt
Improved lossless audio support
Auxio's lossless audio support is somewhat lacking. This is caused by several issues:
- ExoPlayer resamples very high quality 24-bit or 32-bit PCM audio down to 16-bit PCM audio. This kills any quality that a loss-less audio file might have.
- Worse still, the resampling operation and my own ReplayGain engine adds enough overhead as to cause distorted, choppy playback in certain situations, as is covered by #151 and (maybe) #157.
This issue is about some internal technical reworks to fix this. The great thing about ExoPlayer is that you can rip open the player implementation and replace nearly everything with your own code, and that is (to an extent) what I want to do.
Basically, this is my plan (in technical terms):
- Rework the ReplayGain system to handle far more encodings ~~and use the lower level audio processor primitives~~ (Done in c372126).
- Enable float output in the audio sink. Not sure what implications this might have for performance, but I hope it's better than the current resampling strategy. However, this still causes the audio sink to resample audio (just in a different manner), so thus I must also do step 3.
- Override the audio sink to remove the default audio processor chain and replace it with my own. This will be solely composed of the ReplayGain system.
- (Not confirmed) If I implement an equalizer or playback speed support, additional audio processors will need to be implemented that need to be similarly low-level. Playback speed in particular will have to be modified in order to handle more encodings, or resample on it's own.
this is all super cool, but like, do you just know how exoplayer and streaming audio on android works? this all seems like you've been working with android for 20+ years lol. I'm amazed and keep up the good work!
Yeah I basically know the internals of android's media system, however I that's the product of 2 years of developing this app and going "No, that solution isn't good enough, I'm going to find a better way." I've also developed a good habit of not completely relying on external documentation and actually reading the source code of what I use. grep and cs.android.com are life-savers in that regard.
Was going to take a look at this, but see the next-release tag. Has this already been completed in a wip branch not yet pushed?
next-release signifies not that I've completed it, but rather that I'm aiming to work on it for the next major release. In this case, I'm aiming for this, #108, #93, and #85 to be added in a 2.7.0 releases (or 2.6.1). @dreamsyntax
Okay, things have gotten more difficult. I was under the impression that I could side-step ExoPlayer's default float output behavior, which disables all audio manipulation like ReplayGain, by simply replacing the entire audio processor chain. Turns out I cannot do it at all with the public AudioSink API. Any new chain I set will still be appended to the internal array of audio processors, which are disabled when float output is enabled.
Thus, I am left with a few options:
- Make the better lossless audio functionality a setting. I cannot do this, as it is impossible to enable float output or swap out AudioSink implementations at runtime.
- Reflect and replace the audio processors myself. This is probably not a good idea.
- Talk to the ExoPlayer devs about why audio processors are disabled with float output, why resampling is done even with float output, and if it is safe for me to side-step it on my own end. This is probably what I'm going to try doing. If I am lucky, I could PR a change, but if I am not, I'll just have to maintain my own fork with my fixes enabled.
Okay, I made a related issue here. I'll try to start work on a PR eventually so I can properly implement this.
Started the first PR in what's probably going to be a series of changes I'll make to ExoPlayer eventually here. This should allow me to use float audio, but trying to optimize the resampling problem probably needs to be included in another pull.
Actually, I'm not even going to try to upstream this. I think I'm just going to maintain my own ExoPlayer fork with these fixes hacked in for now, any maybe I'll refine it enough in the future so that it's mergable.
Okay, so I can feasibly implement this for the next release. I just don't want to. I've had to mangle so many components that won't play along with non 16-bit PCM audio and I am extremely worried that knock-on effects that might have. I need to put more effort into my custom ExoPlayer fork and rework at least some of the fundamental components just to be safe, and even then.
I think I'm going to shift priority, at least for a time. I don't have enough time to focus on extremely expansive projects like this.