Support audio processing with more than 2 channels

[PhysSong]

Some plugins require more than 2 channels for its audio input/output. Here are some common use-cases:

• Stereo sidechain compressor: requires 2 stereo inputs
• Drum kits: some drum kit VSTs can use multiple outputs for multiple samples, for examples, stereo out 1 for kick, stereo out 2 for snare

However, there are some difficulties which come from the internal structure of the LMMS core:

• LMMS uses interleaved stereo formats while external plugins like VST, LADSPA, Zyn, and LV2 expect non-interleaved format. For audio outputs and file outputs, the interleaved form might be beneficial. However, I think it's not for internal processing.

Features that can be implemented:

• Multi-channel matrix effect, as an extension of the current stereo matrix plugin
• Better support for sidechain compressor or vocoder plugin, see #3817 and #4337
• Native drum sampler(#4075) with per-sample effect mixing

[JohannesLorenz]

The question is also how to visualize that. I find the classical mixer window not suited for more than 2 channels, but I'd prefer a graph where each plugin is a node and you can connect to the inputs by using arrows ("edges"). What do the others think about the visualization?

LMMS uses interleaved stereo formats while external plugins like VST, LADSPA, Zyn, and LV2 expect non-interleaved format.

True, but why is this more difficult with 6 channels than with 2?

[DomClark]

Another issue I can see is fitting this around the instrument's built-in effect chain, which is only able to process a single stream of audio. Should we offer the option of multiple effect chains, one for each stereo channel pair, or should we disable the effect chain for multiple-output instruments, in the same way we disable envelopes for single-streamed instruments?

What do the others think about the visualization?

I like the idea of a graph, and this seems to be a feature that other DAWs offer (FL Studio's Patcher comes to mind, but I'm sure there are others). I don't think this should be a replacement for the current mixer though - it could easily get disorganised and unmanageable for large projects.

[Spekular]

A sane default would be to apply the same chain to each stream, no? Additional flexibility would be great too, of course, but being unable to use effects on a plugin at all seems rather limiting.

[DomClark]

Certainly that would be ideal, but that may be difficult or impossible to implement. Playing each stream through the same chain won't work (imagine, say, a reverb effect where the input comes from a different source every 256 samples...). Having a single UI controlling multiple chains could be tricky, and some plugin types don't support this (in particular, though perhaps not limited to, VST2, and some inflexibly-written VST3 plugins).

Another possible solution may be to add a "simple mode" to plugins with multiple stereo channel pairs, where all the outputs get mixed down to two channels, at which point the audio can be processed as normal.

[Spekular]

Ah, sorry, I meant "the same" as in equivalent. So adding a reverb to the instrument effect chain would "internally" add one reverb for each channel with their models linked to the same view/controller in the UI. I'm probably in over my head when it comes to this topic though so maybe that wouldn't work.

[michaelgregorius]

In Reaper each mixer strip can have an arbitrary number of channels. If there are more than two channels the level meters just become narrower and show the level of each channel.

With regards to the plugins I think one has to take the perspective from the plugins capabilities. Each plugin has a certain number of input and output channels that it can process and output to. Therefore if you put a plugin on a strip you can define which mixer channels feed to its inputs and where the output channels should go.

If you open a plugin in Reaper you can click a button to open the mixer matrix for the plugin:

This opens the mixer matrix:

By default it has the following mappings for the input:

• 1 -> 1
• 2 -> 2

The same applies to the outputs. If you click the "plus" icon you can expand the input and output channels that are shown for selection:

I guess LMMS would need something similar if it wants to support channels with more than stereo information, e.g. surround, sidechaining, etc. If I remember correctly Ardour also provides a mixer matrix.

I think the effect chain of the internal instruments could be kept as it is. I assume that each instrument has no inputs and two outputs. Therefore they are inherently stereo and should be able to deal with stereo plugins because they already know how to deal with them right now.

[messmerd]

Support for multi-channel plugins has been implemented in #7459. I went with @michaelgregorius's suggestion and created a "Pin Connector" based on the one from REAPER.

It required rewriting all of the audio processing code between plugins and the core, but the design I settled on is incredibly flexible as well as efficient, which should allow us to adapt it for use with various plugin formats.

I've already done so for VSTs, allowing the use of multi-channel drum kits. band splitters, etc. And mono VSTs are now handled correctly without pretending they are stereo.

There's still a lot more work to do before proper sidechaining and audio routing are a reality, but supporting multi-channel plugins is a significant first step. The next step should be to add support for multiple "track channels" like in REAPER.

I'm thinking I should open a discussion to outline my game plan and make sure we're all in agreement, then a start a GitHub project or meta-issue to track progress.