LMMS
Refactor PortAudio backend
Applies various changes in an attempt to resolve certain issues with the PortAudio backend and improve the quality of the code.
This PR fixes DirectSound and MME when using PortAudio, but for some reason WDM-KS and WASAPI still lay down broken. This might have to be looked into later as I couldn't find a solution.
How does one test this? Should I play two demo tracks and look at the latency?
Here's my config, @sakertooth, with recent files and directory paths obscured:
<?xml version="1.0"?>
<!DOCTYPE lmms-config-file>
<lmms version="1.3.0-alpha.1.740+g303215f8b" configversion="3">
<MidiJack device="lmms"/>
<app loopmarkermode="dual" nanhandler="1" openlastproject="0" nommpz="0" sololegacybehavior="0" configured="1" displaydbfs="1" disablebackup="0" language="sr"/>
<audioengine mididev="WinMM MIDI" framesperaudiobuffer="256" audiodev="SDL (Simple DirectMedia Layer)"/>
<audiojack clientname="lmms" channels="2"/>
<audioportaudio backend="" device=""/>
<audiosdl inputdevice="" device=""/>
<midi midiautoassign="none" autoquantize="0"/>
<paths ladspadir="..." workingdir="..." stkdir="data:/stk/rawwaves/" defaultsf2="..." sf2dir="..." vstdir="..." theme="data:/themes/default/" backgroundtheme="" gigdir="..."/>
<tooltips disabled="0"/>
<ui saveinterval="2" enablerunningautosave="0" displaywaveform="1" vstembedmethod="none" animateafp="1" smoothscroll="0" sidebaronright="0" trackdeletionwarning="0" printnotelabels="1" mixerchanneldeletionwarning="1" compacttrackbuttons="0" disableautoquit="0" vstalwaysontop="0" oneinstrumenttrackwindow="0" enableautosave="0" letpreviewsfinish="0"/>
<recentfiles>
...
</recentfiles>
</lmms>
Saker claims this is merge-ready, as there is little to be done left.
MSYS compilation yields four backend, with the following results, after some crude testing:
- For WDM-KS, the output is "Could not open PortAudio: Invalid device"
- For WASAPI, the output is "Could not open PortAudio: Invalid sample rate"
- DirectSound works
- MME works
Additionally, with WDM-KS, with certain outputs selected, LMMS won't report the above message, but will instead hang and result in the following output:
Lv2 plugin SUMMARY: 0 of 0 loaded in 0 msecs.
(This is where LMMS hangs after being disabled. Ctrl-C shows the below text)
QObject::killTimer: Timers cannot be stopped from another thread
QObject::~QObject: Timers cannot be stopped from another thread
Checked again!
For WDM-KS, the output is "Could not open PortAudio: Invalid device" For WASAPI, the output is "Could not open PortAudio: Invalid sample rate" DirectSound works MME works
Still true, all of it, including the hanging.
However, 2/4 backends now work well, which I think is great!
Please merge ASAP. 🚀
Please merge ASAP.
GitHub demands your blessing with an explicit approval before I can merge (in general though it needs one approver before merge, at least for this repository in particular currently). I mentioned one last test if you want to take that on though, which is just checking if MSVC builds work for WASAPI and WDM-KS. Here's a link to them for this PR.
Tested with MSVC. Pretty much the same as MSYS2:
- MME works.
- DirectSound works.
- WASAPI doesn't create sound.
- WDM-KS doesn't create sound. It also hangs the program, so it remains active in the Task Manager after the program is closed from the GUI.
Gonna test using MinGW again as well.
Tested with MSVC. Pretty much the same as MinGW:
MME works.
DirectSound works.
WASAPI doesn't create sound.
WDM-KS doesn't create sound. It also hangs the program, so it remains active in the Task Manager after the program is closed from the GUI.
Gonna test using MinGW again as well.
This is truly incredible
Corrected https://github.com/LMMS/lmms/pull/7444#issuecomment-2651562418.
MinGW features only two backends:
- MME works.
- WDM-KS doesn't create sound. Also, unlike MSVC, it doesn't hang the program? Odd!
After trying both, it seems neither of them generate the hanging behaviour, LOL!
In any case, I'd say that PortAudio is fixed, for the most part. It doesn't chop like it used to, and the ability to specify the channel count has been added. I would investigate this further outside of this PR and we can make future improvements to PortAudio.
Yup, I can't reproduce the hanging now, somehow! Must be something to do with running MinGW before/after MSVC, that's my best guess.
@bratpeki asked me on Discord to test macOS support.
- ✅ PASS: Sound output: Testing
demos/unfa-Spoken.mmpz, built-in speakers: Playback is fine - 🚫 FAIL: Testing with Apple Airpods, each airpod shows as a single 1-channel device (sometimes they report as 0-channel and 2-channel respectively), playback is broken, closing and reopening the software will always re-prompt for the device.
- Master shows two devices as well, but playback is possible and it does not re-prompt for the device when closing and reopening
Pushed the last few fixes I had in mind. Will merge soon if there are no objections. Any other potential issues might have to be looked at later.
Pushed the last few fixes I had in mind. Will merge soon if there are no objections. Any other potential issues might have to be looked at later.
They aren't potential issues, they're actually issues because the behavior regresses from master.
Here's what the terminal says:
Could not open PortAudio: Invalid number of channels
No audio-driver working - falling back to dummy-audio-driver
You can render your songs and listen to the output files...
I wasn't able to get this message to reproduce reliably, but it's the closest thing to a usable error I was able to obtain. No matter of configuration will allow the AirPods to work on this PR.
I wasn't able to get this message to reproduce reliably, but it's the closest thing to a usable error I was able to obtain. No matter of configuration will allow the AirPods to work on this PR.
I'm running out of options of what to do. Any luck with the new commits?
Any luck with the new commits?
I had re-tested prior to posting that and it was still broken. If the intent is to fix after merge, let's make sure the issue is opened, assigned and milestoned prior to merging this.
I had re-tested prior to posting that and it was still broken. If the intent is to fix after merge, let's make sure the issue is opened, assigned and milestoned prior to merging this.
I cant merge in good conscious knowing something broke that wasn't broken before. We might have to leave this in the drafts until I can figure out what's going on.
I've made some changes that print out diagnostics, among other things. Let me know if anyone's able to test this for me.
I've made some changes that print out diagnostics, among other things. Let me know if anyone's able to test this for me.
Failed to support PortAudio format: Invalid number of channels
Device range: [0, 5]
Max channel count: 0
Default sample rate: 24000
Output sample rate: 44100
Output device index: 0
Output channel count: 1
Output sample format: 1
Output suggested latency: 0.01
Output host API stream info?: N
Failed to open PortAudio stream: Invalid number of channels
Failed to stop PortAudio stream: Invalid stream pointer
Failed to close PortAudio stream: Invalid stream pointer
No audio-driver working - falling back to dummy-audio-driver
You can render your songs and listen to the output files...
I'm running out of options of what to do.
I did a quick review and the major change I can see is that the old AudioPortAudio.cpp would automatically open both an inputStream and an outputStream if available.
https://github.com/LMMS/lmms/blob/d145f78332efe5ccfdfe95d600a4aeb9bbb96d90/src/core/audio/AudioPortAudio.cpp#L161
The new PR removed the inputStream portion. I'm not sure if this would impact a mic/headphone combo such as the AirPods.
I've added some debug lines to master to see if they shed any light on this issue:
m_outputParameters.device: 1
m_outputParameters.channelCount: 2
m_outputParameters.sampleFormat: 1
m_outputParameters.suggestedLatency: 0
m_outputParameters.hostApiSpecificStreamInfo: 0x0
m_inputParameters.device: 1
m_inputParameters.channelCount: 2
m_inputParameters.sampleFormat: 1
m_inputParameters.suggestedLatency: 0
m_inputParameters.hostApiSpecificStreamInfo: 0x0
Input device: 'Owner’s AirPods Pro - Find My' backend: 'Core Audio'
Output device: 'Owner’s AirPods Pro - Find My' backend: 'Core Audio'
The input device is useless as of right now, so I removed it in order to not waste precious CPU time. I think I have a idea though. Currently we iterate over all available devices regardless of their backend/host API. I think we have to iterate the associated devices with a certain host API rather than going through all available devices separately.
The latest round of commits fixes the bug, audio is back and working with AirPods on this branch.
The UI is still odd though, showing a zero for channels. I'm not sure where this information is derived from, but master seemed to have the correct count in my debug logs here, so I think there may be some room for improvement: https://github.com/LMMS/lmms/pull/7444#issuecomment-2662112079
Ok, I removed my .lmmsrc.xml and this is the behavior with the AirPods:
- The first AirPod listing is likely the input (mic), it can only be set to 0 or 1 channels, it defaults to 0. LMMS will fallback to Dummy Audio if selected.
- The second AirPod listing is likely the output (speaker), it can only be set to 1 or 2 channels, it defaults to 1.
Assuming the second AirPod listing is the correct one:
- If the second one is selected and "1 channel" is selected, it's mono output, as you'd expect.
- If the second one is selected and "2 channels" is selected, it's stereo output, as you'd expect.
The main issue here I feel is that the headphones SHOULD NOT default to mono. I think fixing this would put the PR back in alignment with behavior on master.
(It would also be preferred for the user to be able to determine the difference between input and outputs in situations where their device names is shared, but this is an enhancement to existing functionality and should be tracked as such.)
The mic listing should not be there as this list should only hold output devices. It seems I have a bit more work to do.
The mic listing should not be there as this list should only hold output devices. It seems I have a bit more work to do.
This may be true, but the mics are also there on master, this is not a regression.
Then I'll make a separate list for input devices, which will be unused for now since there's no audio recording.
Linking this comment here as well for completeness: https://github.com/LMMS/lmms/pull/5990#issuecomment-2676379823
I deleted my previous (rather furious) comment as I saw that the discussion started 6 days ago, but I check again and it was picked backed up just recently.
Put simply, I will add input devices back (that currently bear no fruit and take up CPU time and makes users question why LMMS is opening their mic). I have explicitly stated that and @tresf you have given me confirmation that you understood I will add them back. I do not know why this idea is still churning almost a week later. There is a reason why this PR is drafted. I am still working on it, but currently I do not have my laptop with me.