magnum icon indicating copy to clipboard operation
magnum copied to clipboard

Move audio import into Trade

Open mosra opened this issue 5 years ago • 1 comments

Halfway through adding file callback support to Audio::AbstractImporter for consistency with scene and font importers I realized it just doesn't make sense to have audio import separate in the Audio library. Reasons:

  • Audio and the BufferFormat enum depend on OpenAL, while the actual data import doesn't, at all, basically preventing users from using the importer plugins together with, let's say, fmod, SoLoud or other implementations
  • the importers are built for importing just one audio track -- what about playlists etc.? once we add multi-track support, we're even closer to Trade::AbstractImporter
  • complex scenes that have animations usually also have audio data for the animations, audio attached to various objects, having various orientation etc. -- but at the moment, there's no builtin way to connect these two apart from digging manually into importerState() -- and then, if the audio data are embedded in the file, no way of passing those to AudioImporters
  • there's the proposed MSFT_audio_emitter extension for glTF (or this for OGEX), would be nice to have that supported in the Magnum Player, for example

A possible follow-up question could be if Text::AbstractFont should get merged into Trade as well. In my opinion nope, since text rendering is a very specific thing and it's far from "having a buffer with data that you pass to the GPU or the audio card". Besides that, text objects are usually not embedded in scenes (unlike audio) and if such need arises, it can be always done via a format-specific extension.

Things to do:

  • [ ] drop the WIP commit I have here, as nothing there is useful anyway
  • [ ] introduce a new Magnum::AudioFormat zero-based enum
    • [ ] provide its mapping to Audio::BufferFormat
    • [ ] what about all the weird mu-Law things? Supported by the Wav importer, but ...
  • [ ] new Trade::AudioData class, wrapping format, frequency and buffer data
  • [ ] add audioCount(), audioName(), audioForName(), audio() accessors to Trade::AbstractImporter
  • [ ] rewrite AnyAudioImporter and WavAudioImporter to be a part of Trade
  • [ ] start testing audio import on Android (was blocked by #149 until now, with this the AL dependency would no longer be a problem)
  • [ ] rewrite all other audio importer plugins
  • [ ] implement streaming import (will need to be done for #360 as well)
  • [ ] stop depending on Audio and OpenAL in the plugins repo
    • [ ] simplify the CIs
    • [ ] test audio plugins on Android there
  • [ ] think about how can we provide backwards compatibility? Audio::AbstractImporter hooking into Trade? ugh

mosra avatar Mar 09 '19 17:03 mosra

Looking forward to this! (audio streaming)

hsdk123 avatar Nov 15 '19 13:11 hsdk123