Process all inputs: move only streams recording from selected mic

[Bleuzen]

trafficstars

EasyEffects Version

6.2.1

What package are you using?

Arch (easyeffects)

Distribution

Arch Linux

Describe the bug

Hi, currently there is an option to process all inputs, which could be nice since I want to process my mic for all applications, but it is not usable as is. The problem is that, when this option is enabled, EasyEffects moves ALL recording streams to the EasyEffects Source. This also includes streams recording from monitor sources, so one can't capture desktop audio anymore when this option is enabled. Also in case of having multiple mics this is problematic, since recording multiple of them is not possible anymore, since EasyEffects moves all recording streams to the EE Source, which only processes one mic.

Expected Behavior

No response

Debug Log

No response

Additional Information

To solve this, as an idea, instead of (pseudo code):

foreach (inputstream in inputstreams) {
    // move stream to record from EasyEffects source
}

something like this could be done:

foreach (inputstream in inputstreams) {
    if (inputstream is recording from selected mic in EasyEffects settings) {
        // move stream to record from EasyEffects source
    }
}

[wwmm]

The "solution" to this problem already exists. Add the streams for which no effects should be applied to the blocklist. That being said there are some applications like OBS #807 for which the blocklist doesn't work. And unfortunately there isn't a generic way to handle super corner cases like that.

[wwmm]

something like this could be done:

In theory it seems simple but that approach also has undesirable side effects. The default configuration is using the system default microphone. It is totally possible that the user has 2 applications recording from the default mic but he wants effects only for one of them. If we just move to our virtual device anything recording from the default mic just because that is the mic EasyEffects is using there will also be problems.

The current blocklist approach is far from perfect. But I doubt a perfect solution will ever be available considering what is discussed at #807. In general audio applications do not make good use of the tags supported by the audio servers. If they did a more advanced filtering procedure could be implemented.

[Bleuzen]

The "solution" to this problem already exists. Add the streams for which no effects should be applied to the blocklist.

This is not a solution here. I mentioned multiple mics / inputs (also instruments which I don't want to run through EE). As an example: I want to record in Audacity. Instruments / secondary mics without processing, but main mic with processing. Adding Audacity to the blocklist just won't apply any processing at all, so this isn't doing it. I want processing for Audacity, but only when it tries to record from my main mic, leaving other input streams recording for example instruments as is.

That being said there are some applications like OBS #807 for which the blocklist doesn't work.

I noticed that. If EE would automatically enable processing only for streams that try to record from the chosen input this would fix it.

Change logic to not move all recording streams to the EE Source, but only the ones that try to record from the input set here this problem would disappear. No blocklist needed.

In theory it seems simple but that approach also has undesirable side effects. The default configuration is using the system default microphone. It is totally possible that the user has 2 applications recording from the default mic but he wants effects only for one of them.

See no problem here. In this case, users just shouldn't use the "process all inputs" option:

It is even disabled by default. Just to make that clear: For this issue I'm talking about the logic that only should apply if that "process all inputs" option is enabled. Nothing should change if that option is disabled.

The current blocklist approach is far from perfect.

Yeah it is totally useless for the setup I'm talking about.

But I doubt a perfect solution will ever be available

I do understand this. Everyone has different setups and needs. But I think that with my suggestion (only process recording streams trying to record from the selected input) it should fix it for most users. Recording from monitoring devices (like desktop audio) should work again. Handling multiple input devices should start working. Not perfect for anyone, but a nice improvement.

---

Also one thing I thought about, just as an alternative:

Maybe this could be decoupled from the "process all inputs" option. A new option could be added, something like "only care about the selected input device" or "filter recording streams for only the ones recording from the selected input" (should find better naming for it). If this is enabled, EE could even filter the list of applications here:

to only show the ones that are recording from the selected input:

so that, if that new option is enabled, the application list is filtered and looks like this:

Would even suggest that this option is enabled by default because usually you only want to add effects to your input device, not change the input device entirely.

If the apps list would be filtered, the switch would, for the shown applications, actually only toggle effects processing on/off. But if the apps list shows all recording streams (current behaviour), the on/off switch now doesn't only toggle effects on/off, but also moves recording streams entirely unrelated to the EE source, which processes only the input device selected in EE settings.

An effects on/off switch should only toggle effects on/off. Not lead to something entirely different being recorded.

For example one has a recording stream for Audacity running, wanting to record an instrument. EE is setup to process the mic input. Now if Audacity starts recording, it will be listed in the apps list in EE with a toggle for effects on/off. If it is off, it will record the instrument, as setup in Audacity. If a user turns the effects switch in EE on... they expect to only enable effects processing... but no, it will lead to Audacity suddenly recording the mic, something entirely different. This is irritating behaviour.

Since EE can't process multiple input devices in parallel, ones not related to the selected input in settings could just be filtered out (this is the new option I suggested). I think it would be good to enable it by default for more predictable behaviour / user friendliness.

[wwmm]

Ok, let me see if PipeWire is already giving us the information about to which device the stream is connect to.

[wwmm]

I was almost sure this would be a straightforward thing to do but I am not seeing in the output of pw-cli info id, where id is the audio stream id number, to which node it is connected to. I think it will be necessary to iterate over all link objects to find the streams links. It seems they are the only ones with the information about where the stream input/output is connected to.

It is still doable. No as fast as I thought it would be but still doable.

[wwmm]

I think it will be necessary to iterate over all link objects to find the streams links

I am looking at this again and one problem that is going to be a little annoying to handle is that the the signal PipeWire emits with new links information comes some time after the one about the nodes creation (and this is the one where I have to decide if I move the stream or not). I will have to be careful about how to synchronize things...

[Bleuzen]

Would it help to ask pipewires developer in case there is or will be a better solution or do you think it is fine to work with as is?

[wwmm]

Would it help to ask pipewires developer in case there is or will be a better solution or do you think it is fine to work with as is?

I am not sure it makes sense to make changes in PipeWire because of this. It just happens that nodes and links are different objects. It may not make sense in the server to wait for links before emitting signals about nodes creation, destruction or changes because you can have nodes without links. As soon as a stream or filter is created it has no link at all. But is still important to know they exist.

We will have to live with this and find some way to handle this situation. I am not sure how yet.

[wwmm]

This also includes streams recording from monitor sources, so one can't capture desktop audio anymore when this option is enabled.

I wonder if the workaround discussed at #1541 could be extended to improve things here. I did not find a good solution to the limitations I talked about before. But now I know there is a way to detect when a stream is recording from sink monitors I wonder if instead of applying it just to OBS it would not be better to create a configuration option that would allow effects to be disabled to all stream recording from sink monitors.

[Digitalone1]

This also includes streams recording from monitor sources, so one can't capture desktop audio anymore when this option is enabled.

I wonder if the workaround discussed at #1541 could be extended to improve things here. I did not find a good solution to the limitations I talked about before. But now I know there is a way to detect when a stream is recording from sink monitors I wonder if instead of applying it just to OBS it would not be better to create a configuration option that would allow effects to be disabled to all stream recording from sink monitors.

It's better to add the config option, so we don't have to make a specific code for OBS.

I hope that it would also work for pavucontrol, so we could use it to exclude pavucontrol localization.

[wwmm]

It's better to add the config option

If I have time I will try to do this in the next days.

I hope that it would also work for pavucontrol, so we could use it to exclude pavucontrol localization.

As Pavucontrol uses another kind of stream it is probably better to have a different option for it.

[wwmm]

EasyEffects 6.2.6 has been released today and by default it will ignore streams that record from sink monitors. This will help to solve the problem described in this issue but does not fix it entirely yet because streams recording from different microphones than the one used by EE will still be moved to our virtual devices.

I think we can have a workaround for this case if we look at the stream node.target property and ignore the ones that have it set. Not all streams will have it. But the ones that are created with a source device specified should have it.

[wwmm]

I think we can have a workaround for this case if we look at the stream node.target property and ignore the ones that have it set. Not all streams will have it. But the ones that are created with a source device specified should have it.

So much time has passed that node.target has become deprecated and now target.object has to be used. I have updated our master branch with the partial workaround I talked about before. If EE is already running by the time the stream starts it will check if the custom target.object set by the user matches the mic being used by EE.

There are some limitations. If the user sets a custom mic in EE settings when the stream is already running this check is not done again. So streams that where ignored before won't magically come back without restart the stream or EE.

I guess this is as good as it can get... Let's close this issue...