holzschu
Include AudioToolbox codec support within internal FFmpeg (or at least its AAC encoder)
From the bug reports here, I can see that FFmpeg is among the most used features of a-Shell; I use it near-daily myself. One thing has always disappointed me about it, however: while the internal FFmpeg build includes Apple's rather mediocre VideoToolbox codecs, it does not include support for their audio counterparts.
FFmpeg supports being compiled with Apple's AudioToolbox codecs using the --enable-audiotoolbox configure flag (and in fact also supports it as an output device; I have no idea if that works on iOS though), but it is not enabled within a-Shell's internal build.
That is a shame — unlike their video counterparts, Apple's AudioToolbox codecs, particularly its AAC encoder — are of excellent quality (cf. links below). AudioToolbox also includes a slew of other codecs (e.g. MP3, AC-3, ALAC, FLAC, Opus, …many others), though support for them within FFmpeg varies; the majority are unsupported. They are honestly not why I would like to see AudioToolbox support implemented, however.
The (HE-)AAC encoder is what I (and likely many others) would like to see supported the most. It is generally considered to be one of the best audio encoders, regardless of codec, and is even so coveted amongst audio encoding enthusiasts that a longstanding project exists to hook into the AudioToolbox DLL from iTunes on Windows to create a standalone encoder.
Put simply: being able to use Apple's AAC encoder on any iOS device would be fantastic, and it's likely that the capability to do so would drive more people to use a-Shell. Projects like qaac (above) likely would not exist if Apple's AAC encoder were merely equivalent to any other: it really is that much better.
I am aware that FFmpeg includes its own AAC encoder — and that the FFmpeg developers admit that it is of poor quality ("…currently the second highest-quality AAC encoder available in FFmpeg…", referring to Fraunhofer's FDK-AAC as first best; however, that page also says "Apple's encoder is even better than FDK-AAC").
At the very least, FFmpeg also supports its ALAC and AC-3 encoders, but I am not aware of any comparisons between these and FFmpeg's native implementations. As it is lossless, it is hard to see how ALAC could be any better or worse (perhaps it may be more efficient?), and it's certainly possible that Apple's AC-3 encoder is not as good as FFmpeg's. The same could apply to any other codec AudioToolbox supports (except AAC), but the option to use them couldn't hurt if it were to become available.
Regardless, it's AudioToolbox's AAC encoder that would be most useful (IMO). From a licensing perspective, if VideoToolbox can be used, I don't see why AudioToolbox could not.
As always, thank you!
I did try to compile ffmpeg with --enable-audiotoolbox. It uses OSX-specific libraries and function calls and would not compile on iOS.
It would take a complete rewriting to make it work (it's not the same frameworks, the initialization is different, the function calls have different parameters,...)
I did try to compile ffmpeg with
--enable-audiotoolbox. It uses OSX-specific libraries and function calls and would not compile on iOS. It would take a complete rewriting to make it work (it's not the same frameworks, the initialization is different, the function calls have different parameters,...)
Oh, that's a bummer, thanks for trying though. As an aside, did any previous version of a-Shell have the AudioToolbox codecs?
I think I remember them being in an early build, probably back when there was a downloadable WASM ffmpeg — if so, I suppose a (possibly also downloadable) barebones WASM version of ffmpeg that only does audio could be a workaround. I do remember the WASM ffmpeg's performance being miserably slow even on a M1 iPad, and it was very crash-prone, but I'm not sure if I had tried only encoding audio; since it isn't nearly as resource-intensive as encoding video (or worse, both simultaneously), maybe the performance penalty wouldn't matter as much. Cross-compiling under macOS could work if the system's frameworks are the same, even if the developer frameworks aren't (i.e. if compilation fails because the iOS developer header/framework doesn't expose the needed functionality, but it's still present in the system framework anyway).
My apologies if macOS and iOS don't share frameworks as much as they used to though, admittedly my knowledge of both of their internals is a bit outdated. I haven't had a Mac to use for development for about 5 years, so I've kinda lost track of it — you'd think with the ARM transition that they'd be closer than ever, but apparently not.
I do know that the AAC encoder can be used within iOS somehow though — the "Encode Media" shortcut will use it if you select "M4A" as the format — the problem being that it doesn't give you any control over the encoder's parameters. Thanks to a-Shell and ffprobe I can say that it appears to use constrained VBR up to 256 kbps, but the shortcut unfortunately only allows stereo or mono (while the encoder actually supports up to 8 channels and 1280 kbps, and "true" VBR); it also doesn't allow you to choose between HE-AAC and AAC-LC, though of course the encoder supports both.
Quite frustrating to be so close, yet so far.
Edit: As an alternative, if the licensing permits it, maybe the FDK-AAC codec could be included instead? As the ffmpeg documentation says, it is better than ffmpeg's native AAC encoder, just not as good as Apple's. Second best is still better than third...