External audio effects and roadmap?

[YPares]

Hi! I stumbled upon kira while reading stuff on bevy, while trying to reasearch on the topic of audio engines for games that could lead to expressive uses of audio (most simple example I can think of: "In some action RPG, one of the instruments in the current fighting track represents a character. The more that character loses HPs, the more its associated instrument gets filtered and/or the narrower in the stereo field it becomes")

I had 2 questions re. kira:

• Reimplementing various audio effects in Rust ought to be very time consuming. But there are much efforts out there to provide reusable audio DSPs. I see that "Plugin system?" is something you marked in todo.md, do you have plans regarding that? Because, besides full-fledged audio plugins (think VST) which often come with lots of licensing and packaging problems, one very interesting avenue I can think of is the Faust ecosystem ( https://faust.grame.fr/ ) which is a super high level DSP writing language, totally open and permissive. It provides lots of audio generation & processing tools out of the box or as community scripts, and builds for a ton of different backends, including Rust code generation, C libs that you can easily statically link against, or JIT compilation and execution of your DSP script via LLVM.
• I strongly believe not just audio effects but also real-time audio synthesis and note-level (ie MIDI-level) effects have their place in video game music (sure, you're taking a toll on your CPU budget, but you can use that sparingly, and also not all games have to be AAA). Would MIDI and synths be something you would also be interested in supporting down the line?

[tesselode]

The plugin system idea is more about letting people write code that runs on the audio thread and can start sounds, change parameters, etc. That being said, I don't think there's anything stopping anyone from making a Faust or VST host as a Kira Effect.

I would be interested in MIDI support for that good good Nintendo-style dynamic arpeggios. I have no idea what an API for that should look like though. A MIDI player could potentially be implemented as a Sound, albeit one that doesn't actually produce any audio. Realtime synthesizers could also be implemented using the Sound trait.

In summary, I think Kira has the right architecture to support all the ideas you listed. Those features would probably make more sense as separate crates instead of being built directly into Kira.