bugtracker
bugtracker copied to clipboard
maemo-leste
N900: basic voice calls
@pavelmachek suggested we probably need this package https://gitlab.com/libcmtspeechdata/libcmtspeechdata
I don't know what else is needed to play around with this, but let's use this ticket as a tracking ticket for phone calls, since the previous one, #61, is now closed/fixed.
We'll likely (at some point) need the port the pulseaudio module that is supposed to increase audio quality, but let's also see if we can get it to work without.
I briefly tested ofono support in recent image. Following extra steps would be nice:
- [x] Install ofonod by default
- [x] Provide udev configuration in /etc/udev/rules.d/10-nokia-modem.rules:
# Export GPIOs for nokia-modem
DRIVER=="nokia-modem", RUN+="/bin/ln -sf /sys/bus/hsi/devices/n900-modem /dev/cmt"
# Tag modem device for ofono
SUBSYSTEMS=="hsi", ENV{OFONO_DRIVER}="n900", ENV{OFONO_ISI_ADDRESS}="108"
KERNEL=="phonet*", ENV{OFONO_DRIVER}="n900", ENV{OFONO_ISI_ADDRESS}="108"
- [ ] Insert neccesary modules during boot:
cd /lib/modules/4.15.7+/kernel
insmod ./drivers/hsi/clients/ssi_protocol.ko
insmod ./drivers/hsi/clients/cmt_speech.ko
insmod ./drivers/hsi/clients/nokia-modem.ko
At that point, mdbus2 can be used to talk to modem manually, or ofone from unicsy demo can be used to access modem functionality.
I am working on packaging libcmtspeechdata right now. I have it building on my n900 and will add debian packaging somewhere today. Should the binaries it builds just be installed into PATH somewhere? I will work on uniscy_demo next.
And, do you recommend ALSA or Pulseaudio?
Well.. ALSA vs. Pulseaudio -- libcmt should work somehow with both, but neither result is really great... or usable. Pulseaudio should allow two things to play at the same time... so it sounds useful as you want ringtone when music is playing. (But Pulseaudio adds latency etc...)
Put binary somewhere. I don't think it needs to be in PATH, as long as it is started on boot, things should be fine.
Using unicsy_demo's ofone -p I can start and receive phone calls, but don't get any audio yet. I installed pulseaudio and am running ./cmt_pulse as user (with /dev/cmtspeech chmod'ed to 777).
pulseaudio runs as user.
Trying now with alsa, pulseaudio removed & killed:
user@n900devuan:~/libcmtspeechdata$ ./cmt_alsa NFS sucks, version 0.0.1 !!!Scheduler set to Round Robin with priority 99 FAILED!!! CMTSPEECH: nokiamodem_backend: Opening device, libcmtspeechdata v2.0.0. CMTSPEECH: backend_common: CMT Speech Data state machine activated with SSI_CONFIG_REQ. Hardware PCM card 0 'RX-51' device 0 subdevice 0 Its setup is: stream : PLAYBACK access : RW_INTERLEAVED format : S16_LE subformat : STD channels : 2 rate : 8000 exact rate : 8000 (8000/1) msbits : 16 buffer_size : 2048 period_size : 1024 period_time : 128000 tstamp_mode : NONE tstamp_type : MONOTONIC period_step : 1 avail_min : 1024 period_event : 0 start_threshold : 2147483647 stop_threshold : 2048 silence_threshold: 0 silence_size : 0 boundary : 1073741824 appl_ptr : 0 hw_ptr : 0 3.0000 gain, 0 overflows, 0..0..0 initial write: 3072 Hardware PCM card 0 'RX-51' device 0 subdevice 0 Its setup is: stream : CAPTURE access : RW_INTERLEAVED format : S16_LE subformat : STD channels : 2 rate : 8000 exact rate : 8000 (8000/1) msbits : 16 buffer_size : 2048 period_size : 1024 period_time : 128000 tstamp_mode : NONE tstamp_type : MONOTONIC period_step : 1 avail_min : 1024 period_event : 0 start_threshold : 2147483647 stop_threshold : 2048 silence_threshold: 0 silence_size : 0 boundary : 1073741824 appl_ptr : 0 hw_ptr : 0 read = 16 3.0000 gain, 0 overflows, -9900..-9784..-8034 could not fill incoming buffer cc write failed: Bad file descriptor read = 80 3.0000 gain, 0 overflows, -9900..-7397..-2340 could not fill incoming buffer cc write failed: Bad file descriptor CMTSPEECH: ASSERT FAILED: cmtspeech_nokiamodem.c:1639 'desc->bd.pcount == desc->bd.count - CMTSPEECH_DATA_HEADER_LEN' read = -32 read_raw: -32... 3.0000 gain, 0 overflows, -9900..-9900..-9900 could not fill incoming buffer cc write failed: Bad file descriptor CMTSPEECH: ASSERT FAILED: cmtspeech_nokiamodem.c:1639 'desc->bd.pcount == desc->bd.count - CMTSPEECH_DATA_HEADER_LEN' read = 16 3.0000 gain, 0 overflows, -9900..-7505..2112 could not fill incoming buffer cc write failed: Bad file descriptor Writing : 320 bytes 3.0600 gain, 0 overflows, 0..0..0 error writing sink cc, Bad file descriptor CMTSPEECH: ASSERT FAILED: cmtspeech_nokiamodem.c:1639 'desc->bd.pcount == desc->bd.count - CMTSPEECH_DATA_HEADER_LEN' read = 0 Floating point exception
Can you try running it as root?
could not fill incoming buffer cc write failed: Bad file descriptor read = 80 3.0000 gain, 0 overflows, -9900..-7397..-2340 could not fill incoming buffer cc write failed: Bad file descriptor CMTSPEECH: ASSERT FAILED: cmtspeech_nokiamodem.c:1639 'desc->bd.pcount == desc->bd.count - CMTSPEECH_DATA_HEADER_LEN'
Things clearly don't go well there :-(.
I have pulseaudio version. That needs running as root and running with valid $DISPLAY, so it can talk to pulseaudio. Oh and realtime priority, too... and output needs to go to the file, because terminal is not nearly fast enough to keep up.
Sorry, realtime audio is complex :-(.
DISPLAY=:0.0 sudo nice -n -10 ./cmt_pulse -a -v -v > /my/cmtspeech.log 2>&1
You may want to try pa_test, first. It should be simple microphone-to-loopback; helps verifying mixers and latencies...
I couldn't run the pulseaudio one as root, because it could not connect to user pulse. I can set pulse to accept any tcp connection. I will try with valid DISPLAY and nice. How do I use pa_test? (So I don't try to use it wrong)
pa_test should not need any parameters, IIRC. Redirect output to a file, and valid $DISPLAY, for pulseaudio connection.
Do I need to start a call to use pa_test? I will need to do some reading I think. :) Will pick this up later this weekend or next week. Thanks for the help, and working on all of this!
No, pa_test should work without a call.
You might need to set up mixers to get any audio. Example config files are in /usr/share/unicsy/audio.
And if you have a package somewhere, I might attempt to get pa_test and friends to work (but not today).
I have sound with both alsa and pulse, so that should not be the issue. I will package it today and let you know.
I've packaged libcmtspeechdata: https://github.com/maemo-leste/libcmtspeechdata/commit/fce3e63eaeb2e62b0546237b0486ecf2bbc0ce7e
I spent some time using autotools until I realised that it wasn't in use. :D
https://maedevu.maemo.org/leste/pool/main/libc/libcmtspeechdata/
Package libcmtspeechdata can now be installed on armhf
Actually, I just realised I should probably package a few more tools: right now only cmt_{alsa,pulse,dsp} are packaged. You can also checkout the git repo on the n900 and dpkg-buildpackage -uc (or just make)
Do you want me to package everything that is built?
If it is not a problem, it would be useful for me.
I should be also able to make it myself, but I'm a bit busy ATM.
0.1.0 contains three other tools as well:
# dpkg -L libcmtspeechdata /. /usr /usr/bin /usr/bin/alsa_test /usr/bin/cmt_alsa /usr/bin/cmt_dsp /usr/bin/cmt_pulse /usr/bin/loop_alsa /usr/bin/pa_test /usr/share /usr/share/doc /usr/share/doc/libcmtspeechdata /usr/share/doc/libcmtspeechdata/changelog.Debian.gz
Thanks for package. pa_test does not work at all (some kind of deadlock?)... It works ok on debian 8.11.
If pa_test does not work, cmt_pulse has no chance.
Lets try loop_alsa.
sudo killall pulseaudio ./loop_alsa sudo alsactl restore -f /tmp/alsa.playback.speaker sudo alsactl restore -f /tmp/alsa.playback.call
ofono must be already running here, kernel modules must be inserted.
I'm working over ssh.
sudo nice -n -10 ./cmt_alsa -a -v -v > /tmp/delme.log
NFS sucks, version 0.0.1 Enabling audio path cmtspeech_ofono_test: Increasing verbosity to 1. cmtspeech_ofono_test: Increasing verbosity to 2. cmtspeech_ofono_test: Connection established to DBus (1). cmtspeech_ofono_test: priv_add_cb: socket 3, watch 0x547560 (tracking 0x547560)\
after call is placed
poll returned 1 (count:2, cmt:00, dbus:01)
cmtspeech_ofono_test: got message to if:org.ofono.AudioSettings,
member:Propert
yChanged.
cmtspeech_ofono_test: received ofono AudioSettings change, params
name='Active'
cmtspeech_ofono_test: org.ofono.AudioSettings.Active to 1.
poll returned 1 (count:2, cmt:01, dbus:00)
cmtspeech_ofono_test: read cmtspeech event 6.
cmtspeech_ofono_test: state transition 1.
poll returned 1 (count:2, cmt:00, dbus:01)
cmtspeech_ofono_test: got message to if:org.ofono.AudioSettings,
member:Propert
yChanged.
poll returned 1 (count:2, cmt:00, dbus:01)
cmtspeech_ofono_test: got message to if:org.ofono.AudioSettings,
member:Propert
yChanged.
cmtspeech_ofono_test: received ofono AudioSettings change, params
name='Active'
cmtspeech_ofono_test: org.ofono.AudioSettings.Active to 1.
poll returned 1 (count:2, cmt:01, dbus:00)
cmtspeech_ofono_test: read cmtspeech event 6.
Now, it does not work for me. I don't have good SIM inside at the
moment. But I believe it would work if I had.
Except maybe for floating point exception. Is it possible that float behaviour changed between armel, and armelhf?
-- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
Perhaps cmtspeech interface makes some assumptions about userspace (or has only been tested with armel)? The debian that it works on, is that armel or armhf? I can (re)compile with different float abi to test I guess.
It seems we have two separate issues.
pa_test does not work. That one works ok on x86, some kind of deadlock.
Floating point exception. There is a bit too many of those. Likely my fault, and fact that it was not tested on armhf.
Actually, if you get loop_alsa to work, and have a good SIM, cmt_alsa is worth trying. (See above for instructions).
Win! Ok, so so it worked exactly once, but...
issh [email protected]
sudo su
cd /lib/modules/4.15.7+/kernel
insmod ./drivers/hsi/clients/ssi_protocol.ko
insmod ./drivers/hsi/clients/cmt_speech.ko
insmod ./drivers/hsi/clients/nokia-modem.ko
cd /usr/share/unicsy
ofone/ofone -p &
cd /my/libcmtspeechdata
DISPLAY=:0.0 sudo nice -n -10 ./cmt_alsa -a -v -v > /my/cmtspeech.log 2>&1
Pulseaudio must not be interfering.
I fixed Makefile for armhf and fixed division by zero. Not sure if the fix is needed.... I pushed the changes.
Downlink quality is good, which probably means uplink quality is terrible and/or latencies are unusable. Audio is fun :-(.
Note that I'll happily merge your packaging changes into my tree.
Plus, libcmtspeechdata used to be in debian, with 2.X version. [Which is strange, because i remember repairing damaged sources where part was missing; it might be worth comparing the sources.]
So this version of libcmtspeechdata should probably be marked 3.X, to signify that it is newer...?
Thanks - I will try this soon & work on packaging a bit more once it works. Today I will likely not have a lot of time. Side note: pulseaudio would be good to get to work at some point, I also saw webrtc pa plugins - those may help with echo cancellation and other things?
PS: the reason I forked the repo to maemo-leste/ namespace is because that's how we use jenkins - we usually have it build from repos in our orga/namespace.
Yes, pulseaudio is somehow good.
But... this is really tricky, and latency sensitive. Nokia had good reason to have it as an pulseaudio module -- not as normal application.
Echo cancelling is a hard problem, but we have hard and much more serious problems... Like latency and/or dropped samples, and audio being too quiet. You'll see (and hear) soon.
Help of someone with realtime audio experience would be nice. "This is above my paygrade" :-).
Currently, the libcmtspeech* packages in the repositories no longer have the cmt_{alsa,pulse,dsp} tools included
Ah, on further investigation, this libcmtspeech* hasn't been built on beowulf yet.
The one which ended up on my device is actually from Devuan (debian).
@MerlijnWajer we need to actually build the updated libcmtspeechdata in @pavelmachek 's repo. That one is usable. I have already successfully made a phone call with it, and on N900, i can hear the other party really well.
Sadly, the N900 itself can't be heard on the other phone, but some totally random static. Will test further and inform
Question - libcmtspeechdata is available in devuan/debian, but I notice it has very different files. for example, it has a couple of *.so files, while Leste's build does not have them. Do we not perhaps also need them?
Also, there are some few patches in the Debian version which we may want to incorporate as well (e.g. maybe not-so-important ones like fixing typos in cmtspeech_nokiamodem.c, etc.) Of course, the Debian version does not have @pavelmachek 's latest working patch :-)
As an update, pulseaudio-module-cmtspeech is in the repo now and I hope that within the next 1-2 weeks we'll have the various routing bits in place as well.