Voice Receive EndBehavior Ending Despite User Still Talking Sometimes

[dzlandis]

Which package is this bug report for?

voice

Issue description

• Get the bot into multiple servers.
• Run the command to start voice receiving on multiple different accounts on multiple different servers.
• Observe that despite users talking continuously, sometimes the audio is cut short. If you are using my exact code listed below, you will notice it repeating "I'm listening to..." even when the user hasn't finished talking.

Code sample

You can use https://github.com/discordjs/discord.js/tree/main/packages/voice/examples/recorder I suppose buts here's a little brief part of my code where this issue is happening:

    receiver.speaking.on("start", async userId => {
        const user = await client.users.fetch(userId).catch(e => console.log(e));
        if (user.bot) return;
        const opusStream = voice_Connection.receiver.subscribe(userId, {
            end: {
                behavior: EndBehaviorType.AfterSilence,
                 duration: 100
            }
        });
        console.log(`I'm listening to ${user.username} (${user.id}) from "${interaction.guild.name}" (${interaction.guild.id})`);
     });

Package version

0.10.0

Node.js version

v16.13.0

Operating system

Both Windows 10 and Linux (Ubuntu), tried both sodium-native and libsodium-wrappers but I'm using sodium-native right now.

Priority this issue should have

Medium (should be fixed soon)

Which partials do you have configured?

Not applicable (subpackage bug)

Which gateway intents are you subscribing to?

Not applicable (subpackage bug)

I have tested this issue on a development release

@discordjs/[email protected] [email protected] [email protected] (doubt its fixed :P)

[nyapat]

The issue here is that VoiceReceiver#subscribe returns an AudioReceiveStream, and the method is a bit confusing at first. When the stream is returned, it immediately continues code, but that doesn't mean any audio data from the user has been received yet, because the stream works as such

• when receiving a UDP packet of voice data, check if there's any subscriptions for the userId, if not just disregard the data
• if there is; parse, decrypt and then push the packet to the subscriptions
• if there aren't any new packets in the next iteration; append a silence frame depending on the EndBehaviorType
• (and in the example) when the buffer receives a null terminating packet; end the stream and finish writing to a .mp3 file (after piping through an ogg stream)

(I believe, feel free to correct me if anyone knows more specifically)

TLDR: opusStream is not promisifed, it's simply a stream, you can try extending duration (ms) in the subscription options if you want to wait longer before terminating the stream

[dzlandis]

@nyapat or anyone else who may know: I was wondering if anyone knows if scaling can cause issues with Voice Receive. Because I think that is what I am experiencing. I own a bot which operates in many voice channels (about 300 at once) and people run commands which play sounds in the voice channels. Could this be causing the issue I am experiencing with Voice Receive? Even with one voice channel using voice receiving, it seems to be impacted because of the other voice channels the bot is in. In my testing on a smaller version of the bot, my code works fine with Voice Receive and the audio is not cut into small durations, but in production it's being split and chopped up into smaller sections a lot. Could this be due to lag on the bots end with audio packets becoming distorted due to scale? That's the only thing I can think of right now. If anyone has any idea or may know how to fix, that would be awesome.

Logging wise, the issue looks like this: Listening to Dzlandis.... duration 0.2s Listening to Dzlandis.... duration 0.13s Listening to Dzlandis.... duration 0.17s etc...

When it should look like: I'm listening to Dzlandis... duration 2.3s I'm listening to Dzlandis... duration 3.2s

Essentially, longer splices of talking are being shortened into smaller bits. I just want to make it as clear as possible so the issue can be properly addressed :pray:

[nyapat]

@dzlandis I mean, yes, scaling should affect pretty much everything on your bot, but I'm not sure how it would work with voice receive. The behavior you're explaining just means that it's reaching the silence sooner than expected, which could be from scaling or it could be from you stopping talking too soon. Either way it sounds like something that d.js can't exactly fix, if it's scaling, then it's how the API distributes packets at this scale. Like I said before, you can try extending the duration in subscription options for a longer wait time before ending the stream

[haveamission]

You ever find a solution to this?