vircadia-native-core
vircadia-native-core copied to clipboard
vircadia
Often only getting audio output from web entities or user voices (not both).
I just noticed today that videos in Vircadia have not been playing audio for me even though I can hear other sounds in the program. Someone said videos output through a different interface and it's somehow related to Qt and PulseAudio. The problem may be that I do not have PulseAudio on my (Linux) system. I use ALSA, some (few) people use OSS. It's possible that the audio is being routed to another output and I just don't know because there is only one output channel in the audio settings. I did have to manually specify the output interface in my audio settings so perhaps the other channel is the default?
Yeah, so the Webengine has their own audio output that the Vircadia Interface doesn't influence. Not sure if this is a limitation in Qt or if just no one tried to fix that.
If it outputs to the wrong audio device you will need to change that in your OS. On Pulseaudio one would do that with pavucontrol, no idea about any other sound systems though. I do know that our Qt audio can output to Alsa so the Webengine should be able to do the same.
Sorry I cannot be of much help. Linux audio is just such a huge mess. Theoretically the Webengine should just use the default device, but I have no idea if that works without PulseAudio.
My understanding of how ALSA works is that applications providing audio input select their output stream (e.g. my desktop's front panel jack or a default sink). I don't think ALSA has the ability to alter the output stream an application has selected, but it does have the ability to redirect and mix audio streams if you know which ones you want. The issue here is that I don't know which audio output Webengine is using, or if it is actually using one at all. Application awareness may be possible with the JACK Audio Connection Kit-- I will give that a try before randomly mixing audio channels.
That said, even if I am able to work around this by mixing the audio streams, this does seem like something that should be fixed. If separate audio streams are desirable and there is not some Qt limitation, just including a separate output device setting in the audio settings would solve the problem. Like you said, Qt does work with ALSA in general so I don't know why this would be an exception. If they really don't support it in this specific case, we could probably file a bug report.
Just to be clear, the lack of audio was in some video player entity, not a web browser. Maybe they both use Webengine, maybe they're even the same entity, I don't know.
Yes, we don't have a separate video player. Videos are played in a web entity.
I will mark this as confirmed, as it is true that one cannot change the output device and we are leaving that part to Qt and the OS.
I just tried playing audio from a web entity (Youtube video) and it worked fine. I'm going to be away from my computer for a few days but I will try to confirm if this issue persists with video players when I get back. Could someone provide me with instructions to set up a video player? I seem to recall someone said the videos have to be mp4 and they have to be hosted via HTTP without TLS? Do I just point a web entity at that address?
Youtube isn't really any different from other video players, so if youtube works the video player used for movie night should work as well. (Unless the video player itself is broken)
It definitely worked for other people when it didn't work for me in the past. I would like to test it now but, since I don't know how to set up the video player, I'm not sure when I will be able to. I'll update here if/when I am able to pop into a movie night.
I just joined a movie event and confirmed that I am now getting audio from the video player. Unfortunately, I was not getting any audio from people talking... I was getting U.I. sound effects as usual. Maybe the Qt audio stream switched at some point. I'm using a compiled version now (current master) rather than the AppImage, but I believe I confirmed that the problem persisted after I first switched to the compiled version (different build).
I also tried using Vircadia-x86_64_v2021.1.3-Eos.AppImage. With this, I got no audio at all. Specifying output devices had no effect. I definitely got audio using this exact same binary in the past.
I also tried using Vircadia-x86_64_v2021.1.3-Eos.AppImage. With this, I got no audio at all. Specifying output devices had no effect. I definitely got audio using this exact same binary in the past.
This part is very surprising. Can you create another user account, log there directly after system boot (so Xorg runs from this account too) and run Vircadia from there? You mentioned using ALSA, and I remember it having per-account settings.
Compiling QtMultimedia with QML/QtQuick support did not help. Restarting my computer did not help.
Based on ksuprynowicz's suggestion, I tried disabling my system ALSA configuration files, creating a new user, restarting my computer, and immediately logging in as that user to run Vircadia before any other applications. After specifying the output device in the Audio app, U.I. sound effects and user voices worked properly.
Output of aplay -l was the same for both users. The desired controller (ALC1150) registers one analogue and one digital device as PCH. There is also a digital device for my graphics card, which should not be used.
As expected, selecting any of the following devices in Vircadia seemed to work:
- default:CARD=PCH
- sysdefault:CARD=PCH
- front:CARD=PCH,DEV=o
My testing proceeded as follows:
- I imported my normal user's ~/.asoundrc and restarted my computer-- U.I. sound effects and user voices still worked properly.
- I restarted my computer and logged into my normal user-- U.I. sound effects and user voices still worked properly. I then tried playing a Youtube video in one of the rooms at The Hub, which crashed the application repeatably (see #1234 for logs).
- I restarted my computer and logged into the test user (imported ~/.asoundrc was still installed)-- U.I. sound effects, user voices, and the same Youtube video all worked.
- I restarted my computer and logged into my normal user-- U.I. sound effects, user voices, and the same Youtube video all worked (even simultaneously)...
It seems like this issue may be resolved with the current master build, but is likely contingent on some other variables. I don't plan to test the AppImage since the build I compiled works (for now). I'll post here again if I notice it stops working at some point.
I connected for another movie. This time I got audio from the movie (and system sounds as usual) but could not hear someone talking, who other people could. Restarting my interface did not seem to change anything. I also got audio from Soma FM Player.
Today I tried creating a new user account with the same groups as my normal user (except the user's group) and no other configuration. I tested restarting the application many times, usually clearing the "~/.local/share/Vircadia - dev/" directory and sometimes restarting my computer to switch back to my normal user. Both users were executing the same build.
The problems I encountered were:
- Able to hear other users' voices but not web entities.
- Able to hear web entities but not other users' voices.
- Not able to hear either users' voices or web entities.
In all cases, I was still able to hear the system sound effects.
Restarting the computer and switching between users never solved any of the problems, but the problems on each account would be the same as previously on the same account. Hopefully this is isolated per user.
Problem #2 or #3 (I think #2 but not 100% sure now) was often, perhaps always, resolved by deleting my "~/.asoundrc" and also deleting "~/.local/share/Vircadia - dev/".
I am not sure how I was able to resolve the other two problems but I assume it was some combination of:
- Selecting a different audio output device in the Vircadia Audio app.
- Specifying a different audio output for QtWebEngine to use.
- Adding/removing the same "~/.asoundrc".
- Deleting "~/.local/share/Vircadia - dev/".
- Deleting "~/.config/Vircadia - dev/".
- Restarting my computer (unlikely).
- Switching users.
Note that by "resolve" I mean switch to a different one of the three problems. At no point were both audio sources working without at least an application restart in between. Restarting the application without making any of the changes listed above has never resolved any of the three issues.
The apparent need to go through the initial application load, rebuild the domain cache, and find a user who can talk in voice makes this very difficult for me to test.
I have one lead on what might cause issues #1 and #2 but no idea what causes issue #3. Hopefully #3 is just caused by some sort of cache corruption from my testing process, but it did happen multiple times.
Process and results from today's testing:
I Confirmed that QTWEBENGINE_CHROMIUM_FLAGS was not set.
Initial run: > Audio output in Vircadia Audio app was specified as "front", not "default". > Mic was muted by default. Confirmed presence of Problem #3. I quit Vircadia. I moved "~/.asoundrc" to eliminate my user configuration. This changed the active problem to Problem #1. I quit Vircadia.
Proceeding with Problem #1: Deleted "~/.config/Vircadia - dev/". > Audio output in Vircadia Audio app defaulted to "Computer's default audio (recommended)". > Mic was muted be default. > Tutorial/intro world eventually loaded. > I went through the initial configuration promps. I was not getting any audio because I had no "~/.asoundrc". I specified "front" output in Vircadia Audio app. Problem #1 persisted. I quit Vircadia.
Proceeding with Problem #1: Deleted "~/.local/share/Vircadia - dev/". Did not have to go through tutorial/intro. Problem #1 persisted. I quit Vircadia.
Proceeding with Problem #1: Deleted "~/.local/share/Vircadia - dev/". Deleted "~/.config/Vircadia - dev/". I specified "front" output in Vircadia Audio app. > Tutorial/intro world eventually loaded. > I went through the initial configuration promps. Problem #1 persisted.
Proceeding with Problem #1: I specified "hdmi" (effectively "null") output in Vircadia Audio app. I quit Vircadia. Deleted "~/.local/share/Vircadia - dev/". Deleted "~/.config/Vircadia - dev/". I exported QTWEBENGINE_CHROMIUM_FLAGS='--alsa-output-device=front'. > Tutorial/intro world eventually loaded. > I went through the initial configuration promps. I specified "hdmi" (effectively "null") output in Vircadia Audio app. I could not hear web entities. I specified "front" (effectively "null") output in Vircadia Audio app. Problem #1 persisted. I quit Vircadia.
Proceeding with Problem #1: Cold restarted computer (waiting for 10 seconds to ensure memory discharge). Deleted "~/.local/share/Vircadia - dev/". Deleted "~/.config/Vircadia - dev/". > Tutorial/intro world eventually loaded. > I went through the initial configuration promps. I specified "hdmi" (effectively "null") output in Vircadia Audio app. This time, the web entity (on YouTube) displayed the "CLICK TO UNMUTE" button in the top left corner upon interaction. I could not hear web entities. I specified "front" (effectively "null") output in Vircadia Audio app. Problem #1 persisted. I quit Vircadia.
Proceeding with Problem #1:
I could not get speaker-test to work without reinstalling my old "~/.asoundrc".
I simplified my "~/.asoundrc" to only include the following three lines:
- defaults.ctl.!card PCH
- defaults.pcm.!card PCH
- defaults.pcm.!device 0
Two simultaneous instances of speaker-test worked fine.
Simultaneous instances of these commands worked fine:
- speaker-test --rate 48000
- speaker-test --rate 96000
Two simultaneous instances of speaker-test -D plug:front did not work:
"Playback open error: -16,Device or resource busy"
One instance of speaker-test -D plug:front worked fine.
Simultaneous instances of these commands worked fine:
- speaker-test --rate 48000 -D default
- speaker-test --rate 96000 -D default
Proceeding with Problem #1: Deleted "~/.local/share/Vircadia - dev/". Deleted "~/.config/Vircadia - dev/". I exported QTWEBENGINE_CHROMIUM_FLAGS=' --alsa-output-device=default '. I specified "default" (not "Computer's default audio") output in Vircadia Audio app. > This time I connected to The Hub, since nobody was in the previous domain. This changed the active problem to Problem #2. Specifying other audio outputs in Vircadia's Audio app did not break web entity audio: - Computer's default audio (recommended). - default:CARD=PCH - sysdefault:CARD=PCH - front:CARD=PCH,DEV=0 - hdmi:CARD=NVidia,DEV=0 I quit Vircadia.
Proceeding with Problem #2: I opened all the same desktop applications I had open previously: - chromium (three windows) - signal-desktop - element-desktop - irssi Someone was in the previous domain again, so I connected to that one. Confirmed presence of Problem #2. I quit Vircadia.
Proceeding with Problem #2: I deleted "~/.local/share/Vircadia - dev/". I specified "default" (not "Computer's default audio") output in Vircadia Audio app. I reconnected to the same domain and confirmed that the web entity still worked. I quit Vircadia
Proceeding with Problem #2: Deleted "~/.local/share/Vircadia - dev/". Deleted "~/.config/Vircadia - dev/". > Tutorial/intro world eventually loaded. > I went through the initial configuration prompts. I specified "default" (not "Computer's default audio") output in Vircadia Audio app. I reconnected to the same domain and confirmed that the web entity still worked. I connected to The Hub and confirmed that I could not hear users' talking. Specifying other audio outputs in Vircadia's Audio app did not allow me to hear users: - Computer's default audio (recommended). - default:CARD=PCH - sysdefault:CARD=PCH - front:CARD=PCH,DEV=0 I quit Vircadia
Proceeding with Problem #2: Removed "~/.asoundrc". I specified "default" (not "Computer's default audio") output in Vircadia Audio app. I connected to The Hub and confirmed that I could not hear users' talking. I quit Vircadia
Proceeding with Problem #2: Deleted "~/.local/share/Vircadia - dev/". I specified "front" output in Vircadia Audio app. I connected to The Hub and confirmed that the active problem changed to Problem #1. I quit Vircadia.
Proceeding with Problem #1: Reinstalled the three line "~/.asoundrc" previously noted. I specified "default" (not "Computer's default audio") output in Vircadia Audio app. I connected to The Hub and confirmed that the active problem changed to Problem #2. Specifying other audio outputs in Vircadia's Audio app did not change the problem. I quit Vircadia.
Proceeding with Problem #2: I set QTWEBENGINE_CHROMIUM_FLAGS='' I connected to The Hub and confirmed that Problem #2 persisted with all options in Vircadia's Audio app. I quit Vircadia.
Proceeding with Problem #2: Deleted "~/.local/share/Vircadia - dev/". I connected to The Hub and found that the active problem had changed to Problem #3. I confirmed that Problem #3 persisted with all options in Vircadia's Audio app.
Proceeding with Problem #3: Deleted "~/.asoundrc". I specified "front" output in Vircadia Audio app. I connected to The Hub and found that the active problem had changed to Problem #1.
To summarise my findings on the problems:
- Fixed by assigning default output device in "~/.asoundrc" and also specifying QtWebEngine to use the "default" output device.
- Fixed by removing assignment of default output device in "~/.asoundrc" (although this was not necessary before), maybe unsetting QTWEBENGINE_CHROMIUM_FLAGS, and also removing "~/.local/share/Vircadia - dev/".
- Caused (perhaps not exclusively) by deleting "~/.local/share/Vircadia - dev/". Fixed (perhaps not exclusively) by removing assignment of default output device in "~/.asoundrc".
#3 seems like some sort of caching issue.
#1 and #2 seem likely to be caused by some audio stream encoding that is incompatible with my system's automatic conversion for mixing. This could be caused by the syntax Vircadia, QtMultimedia, or QtWebEngine's Chromium build are using to reference ALSA output devices, but it could also not be. I will try to investigate further. If anyone has information about the audio stream used to output user voices, that would be helpful to know.
The voice data is decoded by a part of Vircadia; What plays it I don't know, but it could be played by QT since that is where we get our list of devices from. The QtWebEngine most likely does it's own thing as it is a very isolated and complicated piece of software.
Ok, I will try to figure out how the voice data is handled but it would be very helpful if anyone could tell me where to look.
QtWebEngine seems to leave all audio-related code to the Chromium it has bundled. That's why I was setting the output device with the Chromium flag. Their code seems to be using the correct ALSA device and syntax though. At least for media content.
![stale[bot] avatar](https://avatars.githubusercontent.com/in/1724?v=4 "Published by a pub.dev verified publisher"){kind=small}