sof
sof copied to clipboard
thesofproject
[BUG] How to support 4 channel microphone on Lenovo X1 Carbon 10
Describe the bug Honestly not sure if this is a bug or a feature request - but the Lenovo X1 Carbon 10 has 4 microphones:
- Dmic0 Front
- Dmic0 Rear
- Dmic1 2nd Front
- Dmic1 2nd Rear They are not individually configurable and I'm not sure we have them setup correctly and am looking for guidance on what to do for the next steps
I've done some initial debugging with Jaroslav Kysela (thanks!) and recommendation was to raise the issue here for wider review.
Doing: arecord -D hw:0,6 -f S32_LE -r 48000 -c 4 test.wav (results are arecord -D hw:0,6 -f S32_LE -r 48000 -c 4 test.wav) can see that channels are in pairs (I did some tests turning the PC round and yelling at the back of it and it didn't really make any difference)
I can't mute/unmute the Dmic1 mics, and if I set the levels to 0 it doesn't make any difference If I mute Dmic0 rear then in the wave capture it silences both channels 3 & 4. If I mute Dmic0 Front it silences channels 1 & 2
Jaroslav noted: The 'Dmic0 Front' and 'Dmic0 Rear' controls are not supported in the current UCM configuration. Also, it's a question why the controls do not follow the PCM channel layout (4 -> 4, it's 2+2 -> 4 now).
Everything is 'working' - but I wanted to look into how we took advantage of this HW configuration and offer the best audio experience to users.
To Reproduce Steps to reproduce the behavior: (e.g. list commands or actions used to reproduce the bug)
Reproduction Rate 100%
Expected behavior Not sure really :) Hoping an audiophile can guide me on what we should be aiming for
Impact Small
Environment alsa-info.txt
Let me know if you need more specifics
Thanks Mark to create this issue. The UCM configuration does not include the support the Dmic Front/Rear volume and switch controls at the moment. We need to know the mapping of those controls to the PCM stream channels.
Could we get more details about this?
@mrhpearson the main problem is that we have no idea at the driver level which microphone is placed where. All that the driver and firmware do is provide a 4ch stream. UCM would need to have a channel map to figure out which mics is which, before we can control individual mics in an idiot-proof way.
For now the usage is more of an 'all or nothing', where all mics gets the same volume and mute/unmute commands. Individual control is tricky without geometrical placement information.
The 'all or nothing' does not explain the control by pair. I would assume the single value control to do this. It appears that two stereo (double channel) controls are in the way.
my point @perexg is that the pairs are completely random, depending on how the board is routed. Even if it worked the usage would be questionable at best.
I don't follow this. The attenuation/gain is implemented in firmware, right? DSP sends the 4ch PCM stream, so where's the issue to do a 1:1 map between this volume control and the channel in the PCM stream? I'm not talking about the physical mic placement.
Got it - I'll see if I can get a channel map then. I assume I can get that from schematics? Thanks for the responses!
I am talking about physical mic placement. If you want a 'front' and 'rear' mixer control, you need to know what the front and rear is. The driver and firmware don't have this information.
We are not on the same track. Basically, UCM needs to define an abstraction for the PCM devices and associated hardware volume control. It means that the volume control should match the number of channels in the PCM stream. My point is that SOF defines 4 channel PCM stream for microphones, but there are multiple stereo controls to set the switch / volume for this 4 channel stream.
I have a polite question, why there is not a 1:1 mapping between the switch / volume controls and the PCM channels. I don't take care about the physical connection at the moment. It means that the switch and volume controls should have 4 values associated to the PCM channels.
I can do a remap for controls inside alsa-lib, but I think that SOF may have a way to simplify things and offer the 4 channel volume controls for apps directly.
Last thing is that it appears that almost no-one use those new volume controls, because the current UCM configurations use just the Dmic0 controls: https://github.com/alsa-project/alsa-ucm-conf/blob/c802ca0b7d8186413b728382f8310359afa6a144/ucm2/Intel/sof-hda-dsp/HiFi.conf#L35-L50 . So I assume that the default value is the max ?
Last thing is that it appears that almost no-one use those new volume controls, because the current UCM configurations use just the
Dmic0controls: https://github.com/alsa-project/alsa-ucm-conf/blob/c802ca0b7d8186413b728382f8310359afa6a144/ucm2/Intel/sof-hda-dsp/HiFi.conf#L35-L50 . So I assume that the default value is the max ?
@singalsu is the default value the max ? @perexg are you also thinking about exposing the Mic positions via ALSA ? Fwiw, Seppo has a working beamformer and the FW has no idea of the DMIC positioning at boot. IIUC, the driver passes this position data in today via topology (which can couple a topology to a particular device), but having some official/consistent API would be good here.
Not sure if it's very helpful...but I got this back from the HW team:
4 microphones are connecting to Intel PCH directly. Intel D-mic driver delivers 4 channel signals to upper layer. Apps are in charge of mixing the multiple channels.
So I'm guessing these shouldn't be treated in pairs and should be 4 individual controls. Let me know if there's any other details I can get that would be helpful.
so I would personally think it should be 1pcs 4ch alsamixer volume, where you can possibly change the individual channels if needed... and overall volume with just with general command. I remember there was some limitation of specifying or just displaying 4ch volume control in alsa/alsamixer. I think I got it sometimes visible but needed to hack around in asoc etc.
My point is that SOF defines 4 channel PCM stream for microphones, but there are multiple stereo controls to set the switch / volume for this 4 channel stream.
I remember there was some limitation of specifying or just displaying 4ch volume control in alsa/alsamixer. I think I got it sometimes visible but needed to hack around in asoc etc.
I also vaguely recall some sort of framework limitation.
In practice even if we had a better volume control at the driver/firmware level, we would still have issues in terms of user interaction. PulseAudio and PipeWire will do a matrixing operation to downmix to 2ch without any user control. @kv2019i worked on this a long time ago and we are clearly very far from having good support for anything > 2 channels in the Linux stack.
I think that we should start somewhere and the first level is the driver (and firmware in this case). Note that USB driver offers the multi-channel controls right now:
Simple mixer control 'PCM',0
Capabilities: pvolume pswitch pswitch-joined
Playback channels: Front Left - Front Right - Rear Left - Rear Right - Front Center - Woofer - Side Left - Side Right
Limits: Playback 0 - 87
Mono:
Front Left: Playback 87 [100%] [0.00dB] [on]
Front Right: Playback 87 [100%] [0.00dB] [on]
Rear Left: Playback 87 [100%] [0.00dB] [on]
Rear Right: Playback 87 [100%] [0.00dB] [on]
Front Center: Playback 87 [100%] [0.00dB] [on]
Woofer: Playback 87 [100%] [0.00dB] [on]
Side Left: Playback 87 [100%] [0.00dB] [on]
Side Right: Playback 87 [100%] [0.00dB] [on]
The alsamixer is capable to split the multichannel to 2ch / 1ch bars and I don't know any issues with amixer regarding the multi-channel support. Writing that, the currently used smixer layer in alsa-lib uses fixed positions, but it should be easy to add the remap from TLV or alsa-lib configuration file.
PA/PW (any app) should receive the correct position information for the improved (customizable) mixing. Eventually UCM can do the stream remix now (if wanted).
I would propose:
- create a 4ch controls for dmics
- improve alsa-lib (smixer API) to allow to override position information (for UCM) - I'll try do it
- extend ALSA control API to allow pass position information (TLV extension - it should probably follow PCM chmap)
The USB specification offers a large list of positions which was partially adapted by @tiwai (PCM chmap) : https://github.com/alsa-project/alsa-lib/blob/v1.2.7/include/pcm.h#L573-L610 . We can add missing positions, of course.
Obviously happy to help out with this however I can. My audio knowledge is sadly lacking but I would love to learn along the way.
@perexg - you should have access to some X1C10's thru RH, but if you have trouble getting hold of one let me know and I'll see if I can find you one.
I would propose:
- create a 4ch controls for dmics
- improve alsa-lib (smixer API) to allow to override position information (for UCM) - I'll try do it
- extend ALSA control API to allow pass position information (TLV extension - it should probably follow PCM chmap)
The USB specification offers a large list of positions which was partially adapted by @tiwai (PCM chmap) : https://github.com/alsa-project/alsa-lib/blob/v1.2.7/include/pcm.h#L573-L610 . We can add missing positions, of course.
@perexg we would be happy to help test here. @singalsu I guess you have some development boards or laptops that have 4 DMICs where you know the positions ?
@singalsu I guess you have some development boards or laptops that have 4 DMICs where you know the positions ?
Yep, I have one with four microphones so I can contribute the data for it and test it.
The USB specification offers a large list of positions which was partially adapted by @tiwai (PCM chmap) : https://github.com/alsa-project/alsa-lib/blob/v1.2.7/include/pcm.h#L573-L610 . We can add missing positions, of course.
This list from USB3 only makes sense for multimedia playback. It doesn't make much sense for a microphone array capture. Most descriptors I am aware of borrow heavily from another description that was initially used for USB mic arrays. https://docs.microsoft.com/en-us/windows-hardware/drivers/audio/microphone-array-geometry-descriptor-format
I think this was also used in the HDaudio NHLT descriptors, along with sensitivity information if I am not mistaken. @singalsu might be able to help here.
As summer wraps up (at least in one hemisphere) thought I'd send in a nudge and confirm what are the next steps here. Is there anything a noob/idiot could get started with or is it best to wait for experts to help with?
Looks like the hardest bit is improving alsa-lib. Are there any suggestions on where I could get started with that?
Mark
There are two separate issues:
- correlate the control values with the stream channels - the driver should do this (actually it's a mix of stereo controls mapped to 4ch PCM stream lacking a solid connection)
- describe the control value / stream channel positions
The second layer is optional. I start work on it next month. But it's not the root of the current problem.
If I can offer a second test platform. The Intel NUC 11 Enthusiast also has a DMIC array (4 ch., all front, ALC700) that defaults to a single microphone in Fedora 36 and can---after some reconfiguration---show input as one of two stereo pairs. The exact method of doing this has been forgotten.
I have the same goal as Mark: four independent microphone channels if possible, or perhaps user selection of a specific, individual microphone.
alsa-info.txt I uploaded my alsa-info, although DMIC does not show up as it does in the above alsa-info. Let me know if I can test proposed changes or submit anything helpful.
When I try 4 channel recording, I get an error arecord: set_params:1396: Channels count non available. I can run arecord -D hw:0,0 -f S32_LE -c 2 test.wav but the outer microphones are disabled:
(To explain the image, I brushed each of the 4 mic openings twice from R to L and then did the same thing again. Small peaks across both channels are when I brush disabled microphones.)
hmm I have the X1 (updated ubuntu to latest), and can see the 2 dmic controls in alsamixer. I see that I have my own commit for this c8849c74912f863edafc69dc2746d63e4fb15d86 in driver. Now trying to remember why I did this, I think the dapm was restricting me to create 4ch mixer... and that the fix was not very easy (it was touching files all over the place).
butbut, in amixer I can see this as 4ch mixer, but alsamixer shows this as 2 separate mixer elements. So I'm actually wondering do I need to do something :)
The alsamixer remaps multi-channel controls as stereo representation. Scratching my head - it seems that there are really 4 values for Dmic controls (using the alsa-info.sh output from the description), so I wrongly interpreted the Mark's observation based on the alsamixer:
Simple mixer control 'Dmic0',0
Capabilities: cvolume cswitch
Capture channels: Front Left - Front Right - Rear Left - Rear Right
Limits: Capture 0 - 70
Front Left: Capture 70 [100%] [20.00dB] [on]
Front Right: Capture 70 [100%] [20.00dB] [on]
Rear Left: Capture 70 [100%] [20.00dB] [on]
Rear Right: Capture 70 [100%] [20.00dB] [on]
Simple mixer control 'Dmic1 2nd',0
Capabilities: cvolume
Capture channels: Front Left - Front Right - Rear Left - Rear Right
Limits: Capture 0 - 70
Front Left: Capture 68 [97%] [18.00dB]
Front Right: Capture 68 [97%] [18.00dB]
Rear Left: Capture 70 [100%] [20.00dB]
Rear Right: Capture 70 [100%] [20.00dB]
Could you verify, if pulseaudio controls all 4 values for those controls when the input volume is changed?
Also, the 4 channel capture should be analyzed using audacity or so...
Also, next question is what pulseaudio does with this multichannel input. If the last two channels are dropped, then the volume may be low.
so I tried first pulseaudio recording (not sure the command line is fully correct though): parecord -d 6 --channels=4 --no-remix --no-remap test_rec.wav
So I'm scratching all 4 mics separately.
Audacity result:
Then stopped pulseaudio and did alsa recording: arecord -Dhw:0,6 -f S16_LE -c 4 -r 48000 test_rec_alsa.wav
Audacity result:
So yes, something is happening inside pulseaudio with volume, or channel mapping or something...
@PennRobotics looks you dont have sof driver or topology loaded... I think. So just saying this is probably different driver code and problem than with X1. So this is not probably using intel dmic hw, but the realtek codecs dmic.
actually now tested again: parecord -d 6 --channels=4 --no-remix --no-remap test_rec.wav
And seem to get all 4 channels as with arecord!
I checked that the ubuntu sound UI rec volume (single slider) also changes all 4 dmic channels volume (checked from amixer).
So not sure did I mess up the sink number etc in my previous post, or did it start to work when I restarted pulseaudio....
At least now this seems to work for me as expected. I'm not sure what happens if one changes volume from alsamixer's dual stereo sliders, it might mess up things.
@PennRobotics looks you dont have sof driver or topology loaded... I think. So just saying this is probably different driver code and problem than with X1. So this is not probably using intel dmic hw, but the realtek codecs dmic.
That's embarrassing. Thanks for the insight. I'll comb through the docs this weekend and see if I can find a valid topology description and get the sof driver running.
Hi @juimonen - I tried that on mine and I see only two channels. I assume because I've not got the patch (c8849c74912f863edafc69dc2746d63e4fb15d86). Dummy question - is that a kernel patch and which tree is it in? Would love to give it a go, and anything else I should pick up :) Thanks though - cool to see it working
@mrhpearson sorry I'm observing our internal sof tree. At least in Linus' tree this is commit fca18e62984a0d797da8379a422a6bb644d68244, "ASoC: SOF: control: override volume info callback", so it should have already been in many kernel versions, I think.
@mrhpearson what distro are you running on your x1? and have you modified the audio stack? I personally just upgraded my 18.04 ubuntu to 22.04 lts, didn't do anything else...