[SFOS] Prevent voice navigation stopping other audio

[paulvt]

If I am listening music while navigation, the music stops after each voice navigation announcement and I have to resume it again (while not paying attention to traffic). This probably also ties into #104.

There are a few options here, I guess:

• Let voice navigation fail/be dropped if there is audio
• Somehow "free" the audio channel after the voice announcement has been made (if possible)
• Dim the current audio, play the voice announcement and undim it again (best way, I'd say)

As a last remake: Pure Maps is an awesome application, almost got all, amazing!

[paulvt]

I have noticed that also a lot of other applications struggle with this, while the core Sailfish OS applications don't really. It might be that the latter have custom handling for this (e.g. music is dimmed for a short while to play an "incoming message" sound for example) through PulseAudio?

[rinigus]

Its not a priority for me right now, but I will look into it when time permits. If someone wants to investigate the issue and propose a solution, that would help a lot.

[bonanza123]

@paulvt maybe that's a thing to ask the jolla devs on TJC or the in the community meetings?

[paulvt]

@bonanza123 Yes, I was considering this too. Had no time for it yet, though!

[theyosh]

Hmm, I looks like quite easy:

Change: https://github.com/rinigus/pure-maps/blob/e84bf7f95cb4b881bdec68dbee7f700c3364bb9b/qml/pure-maps.qml#L71 from Audio to SoundEffect https://doc.qt.io/qt-5/qml-multimedia.html#soundeffect

And remove the lines with autoLoad: true loops: 1

Those do not work. Then, when I have my media player running, the voice will play over the music. Only there is no volume option. So the voice is not always loud enough.

So when we need to crank up the volume, the voice generators should produce WAV files with higher volume in it.

[rinigus]

Is the SoundEffect quieter than Audio? I cannot increase WAV volume, at least using some switch for flite/mimic. Sounds line SoundEffect requires more resources.

Otherwise, we should surely test it. I don't have anything playing on the background, so cannot test that aspect.

[paulvt]

Does that not depend on the relative volume settings for media and notifications/ringtone in the OS? I mean, I sometimes have that notifications are softer wrt the music that I am listening and the other way around. I might be wrong here. At least, on the desktop PulseAudio keeps track of these volumes for the user.

[rinigus]

Good question - I don't know what is it linked to.

[theyosh]

Hmm, I could only find a ringtone volume setting, no notification volume setting. And when the media player is running, the volume buttons it will increase the playback volume. And I guess that the SoundEffect will just use the same volume. I do get the feeling that when a notification sound is played, the media player volume is reduced... but I could also imagine that...

Also I have not investigated the difference between Audio and SoundEffect object and their limitations. The new multimedia library in QT should also support 'types of audio', which has a notification type and voice command type. But that did not seem to work on my Sailfish OS. https://doc.qt.io/qt-5/qml-qtmultimedia-audio.html#audioRole-prop

So then I turned to the SoundEffect, and that just worked out of the box. With some minor changes in the QML file.

So, now I have found a way to increase the Voice volume. I was able to install ffmpeg and ffmpeg-tools on my Sailfish OS, and then used ffmpeg to increase the volume: https://trac.ffmpeg.org/wiki/AudioVolume

So, in order to make the audio crank up logic work for all Voice generators, I added the following code to the 'voice_worker' function at https://github.com/rinigus/pure-maps/blob/master/poor/voice.py#L220

if fname is not None:
          tmpfile = fname + ".tmp.wav"
          shutil.move(fname,tmpfile)
          subprocess.call(["ffmpeg",
                     "-i", tmpfile,
                     "-filter:a", "volume=11dB", "-y",fname])
       	  os.remove(tmpfile)

ffmpeg will take only miliseconds to process the WAV files. And you could make a setting for the amount of dB, so that you can make an adjustable Voice volume setting.

I test and made this code just by changing you app code on my device directly. That works the easiest for me. So no pull request at the moment

[rinigus]

I looked into audioRole as well, but it was a failure for me too. As for using ffmpeg - interesting. It adds one more dependency, but its probably OK. I'll have to check it, give me couple of days.

So, we are also looking for solution of #16 , which I closed since I couldn't see additional option on generator. Good job!

[theyosh]

Cool! Did not see that. :) As I did some testing, the value 11dB is the max value. If you go higher, it gets awful. Also, there is a 'loudnorm' option, which does its job, but had the feeling that was not loud enough for me. But it could maybe be added as an option somewhere at sound settings or voice settings.

[theyosh]

Also, ffmpeg itself is installed automatically. According to my zypper:

zypper info ffmpeg
Loading repository data...
Reading installed packages...


Information for package ffmpeg:
-------------------------------
Repository     : jolla                                     
Name           : ffmpeg                                    
Version        : 4.1.1+git1-1.2.1.jolla                    
Arch           : armv7hl                                   
Vendor         : meego                                     
Installed Size : 7.9 MiB                                   
Installed      : Yes (automatically)                       
Status         : up-to-date                                
Source package : ffmpeg-4.1.1+git1-1.2.1.jolla.src         
Summary        : FFmpeg video encoding and decoding library
Description    :                                           
    FFmpeg: a complete, cross-platform solution to record, convert and stream audio and video.

But you need the ffmpeg-tools to get the ffmpeg binary.

zypper info ffmpeg-tools

Loading repository data...
Reading installed packages...


Information for package ffmpeg-tools:
-------------------------------------
Repository     : jolla                            
Name           : ffmpeg-tools                     
Version        : 4.1.1+git1-1.2.1.jolla           
Arch           : armv7hl                          
Vendor         : meego                            
Installed Size : 300.6 KiB                        
Installed      : Yes                              
Status         : up-to-date                       
Source package : ffmpeg-4.1.1+git1-1.2.1.jolla.src
Summary        : FFmpeg tools package             
Description    :                                  
    Development tools for FFmpeg - a complete, cross-platform solution to record, convert and stream audio and video.

And they are available by Jolla itself, so I think you need only to depend on ffmpeg-tools. But that should be tested, as I have already installed it.

[theyosh]

Hi,

is it at least possible to update the code in https://github.com/rinigus/pure-maps/blob/master/qml/pure-maps.qml#L76 to:

SoundEffect {
        id: sound
}

basically, that is the answer to this issue. It is not the nicest option, it does do the job. Now I have to patch that. For now, the volume is secondary. I do like my music more.... ;)

[rinigus]

I will have to test it again, but I think that SoundEffect was not played via Bluetooth on my device. It could be down to the port or general SFOS feature. But let me test again to be sure

[rinigus]

Tested it finally. On my device, there are issues with Bluetooth (BT) switching when using SoundEffect that are not there with Audio. Namely, while running Pure Maps:

• on phone speaker, test is OK
• switch over to BT, test is OK
• swich off BT, test is silent. Get test working after restart of Pure Maps

For Audio, I haven't seen this issue. Tested using Testing feature in Pure Maps preferences. So, as it is, SoundEffect is not really a reliable solution

[paulvt]

When applying the change mentioned above and testing it with my BT headset I have different test results:

• on phone speaker, music plays, notification plays through music, test OK
• connect headset, music plays on headset, notification plays through music on headset, test OK
• disconnecting headset, music stops, notification plays (over nothing), test OK?

So maybe something changed in Sailfish OS in the meantime (they noticeably did fix some BT/audio routing stuff for me), our tests are different or something else is going on?

On a different note, I was confused by the discussion about the notification volume, because I had assumed that using the "sound effect" style, it would work the same as an incoming sms/XMPP message notification, namely that the media volume is slightly dampened for the duration, but that does not seem to be the case, that is very unfortunate! So, I still wonder how they do that in.. Lipstick?

[rinigus]

I will retest with the latest SFOS with SoundEffect, I don't understand your third test result. Did you hear the notification over speaker? I presume you tested it with SoundEffect, right?

[paulvt]

Yes, with SoundEffect patched in the QML and yes, sorry, the notification was still sent over the speaker, but not over the music, as that is stopped when one disconnects the BT headset.

[padu22]

After struggling with this on UT, I've opened a thread at the UBports forum sound issue playing music and voice navigation. There was a comment about PA providing "alert" audio streams that damp all other audio. Maybe that helps?

[rinigus]

Pure Maps is using QML Audio for sound. We tested SoundEffect as well on SFOS, but it had some issues with the bluetooth for me - will have to retest it. I don't know how to patch it to PA alert. You could see if SoundEffect will work for you (change it at https://github.com/rinigus/pure-maps/blob/master/qml/pure-maps.qml#L88) and whether it works better.

[padu22]

Reading QTMultimedia docs, I stumbled upon Audio property audioRole, which provides a value AlarmRole. So, I added the line

        audioRole: AlarmRole

to the Audio section, but when trying to run this on Ubuntu Touch (UT), the app refuses to start with the following log message

QQmlApplication failed to load component

file://opt/click.ubuntu.com/pure-maps-slim.jonnius/1.28.1/share/pure-maps-slim.jonnius/qml/pure-maps-slim.jonnius.qml:92
"Audio.audioRole" is not available in QtMultimedia 5.2.

A web search indicates that property audioRole might have been introduced in QT 5.6. Would it be possible to update that component? (Please forgive my ignorance, I have no idea whether that component is actually bundled with Pure Maps or provided by my OS (UT)).

[rinigus]

@padu22: try to bump https://github.com/rinigus/pure-maps/blob/master/qml/pure-maps.qml#L20 to 5.6. It is part of UT for you.

[padu22]

Thanks! Pure Maps starts indeed again when using audioRole: AlarmRole and requesting QTMultimedia 5.6. Yet, again I find a log message

file://opt/click.ubuntu.com/pure-maps-slim.jonnius/1.28.1/share/pure-maps-slim.jonnius/qml/pure-maps-slim.jonnius.qml:92:
ReferenceError: AlarmRole is not defined

Obviously, I'm stabbing in the dark.

[jonnius]

I think it should be Audio.AlarmRole instead.

[padu22]

I think it should be Audio.AlarmRole instead.

That's it! With Audio.AlarmRole other audio streams are damped. Though, alarm audio stream seems to be played at heart attacking maximum volume, regardless current audio volume. Audio.NotificationRole seems to be more appropriate.

[rinigus]

@padu22, what about adjusting volume by buttons when using notificationrole?

[jonnius]

You can use uVolMan to control the different volumes btw.

[padu22]

Have to get an API key first. Test voice is too short to do proper testing. Will report back.

[jonnius]

You can test with OSM Scout Server. No need for an API key then.

[padu22]

While the test voice works, I get no voice directions when doing true navigation, whether I use audioRole or not. I'm on the slim version with Pico TTS only, currently. Restarted device multiple times. The official full version installed in parallel has voice output. Any advice?