Invalid Participant Removed Update from Large Gallery Participant

[1fabi0]

Following Scenario causes an issue in policy recording solutions with large gallery I am currently working on:

Initial situation without bot We have a meeting with 12 participants all main video enabled streaming an image into the meeting, without large gallery in use.

Joining situation of the bot

• Now we add the bot into the meeting either by policy, with an additional user under compliance recording policy, or manually via ICommunicationsClient.Add()-method.
• The bot establishes the call to the meeting. (It also waits that the call has state established)
• [The bot sets recording status] // Optional
• The bot uses the ICall.AddLargeGalleryAsync()-method to add the large gallery into the meeting.
• The bot receives a participant collection update that a large gallery participant was added(the large gallery participant that was added has a main video media stream with direction SendReceive and is sending a valid video)
• The bot receives a participant collection update that another large gallery participant was added(this large gallery participant has IsOnHold-Property set to true and the video media stream is set to ReceiveOnly)
• The bot receives a participant collection update that the large gallery participant that is sending video was removed from the participant collection.
• [If we try to readd the large gallery we get error code 400, large gallery already added] // Optional

Workarround to receive large gallery video, after the large gallery that is sending was removed To still receive large gallery video we can keep a video socket subscription to the msi that was sending the large gallery video, even after the large gallery participant, from which we got the msi we are subscribed to, was removed from the participant collection.

Summary The bot receives the notification that LargeGallery was removed from the participant collection of a call but can still receive the large gallery video of the removed large gallery participant.

Example Call ID 1c003a80-15dc-43de-ae37-1c87d2580c93

I do not want to share any more logs as they might expose data that I do not want to share in public.

@samarthasthana @Hunterjet @ssulzer

Can you please explain why the initial LargeGallery participant is leaving the call? A proper solution would be greatly appreciated, as the current workaround seems very unstable and adds a lot of difficult to understand code paths for just recording a simple LargeGallery participant.

[1fabi0]

What is the status on this issue? It has even gotten worse because sometimes the large gallery doesn't even continue streaming the video after the sending instance left.

In non-compliance recording scenarios, removing the OnHold gallery view is an option to add a new gallery, but in compliance recording, this is not possible (It fails with an error message).

So how can we properly record LargerGallery videos in a compliance recording use case?

@samarthasthana @ssulzer @Hunterjet

[yk-recall]

Hey 1fabi0, if you're encountering issues with policy recording solutions in large gallery view, specifically where the bot receives a notification that the LargeGallery participant was removed from the participant collection but can still receive the large gallery video, you might want to check out the Recall.ai API.

It’s a simple 3rd party API that lets you use meeting bots to get raw audio/video/metadata from meetings without you needing to spend months to build, scale, and maintain these bots.

Here are the API docs:

https://docs.recall.ai/

[1fabi0]

@yk-recall can you please explain me how recall managed to avoid this issue, it seems to be a plattform issue that I think is not only related to customer solutions please explain me how recall could help me in any way

[InDieTasten]

@1fabi0 I think @yk-recall just used an LLM to hallucinate some marketing text without even understanding what the issue is.

[ssulzer]

Hi @1fabi0 Can you share a more recent example call-id please.

[1fabi0]

Hi @ssulzer I have the following call Ids for you:

• 17003e80-fffe-4a22-a0a0-7b5a672b0967

This is a meeting with 12 participants one of them under policy the participant under policy joined last. The recording bot invited the large gallery. The large gallery joined(participant id: a2829b99-ab2d-43b6-bcdf-c524cbfd1490) and shortly after left again, but the large gallery continued to stream. Also a second large gallery participant joined at the same time as the first left (participant id: 007a001d-076a-4ea2-8c34-2be5c58f283c), but the new participant does not have the capability to send video/the main video socket is receive only.

• 46003380-ed99-417b-bea7-c3b5d5aca369

This is from the same meeting but the user under recording policy left for a short time and joined again. For the new recording bot instance only the large gallery with participant id 007a001d-076a-4ea2-8c34-2be5c58f283c joined. The large gallerys main video socket is still receive only

[ssulzer]

Hi @1fabi0 we are taking a look. The Large gallery bot will usually join multiple instances of itself into the meeting (each with its own participant-id), but both instances stay in the meeting together, the first one doesn't leave when the second joins. How is your bot observing that the first instance (in this case participant id a2829b99-ab2d-43b6-bcdf-c524cbfd1490) leaves when the 2nd instance (007a001d-076a-4ea2-8c34-2be5c58f283c) joins? (The first Large gallery instance deliberately sent another instance of itself into the meeting to help out. :)

[1fabi0]

Hi @ssulzer In the case above I was able to validate that the first Large gallery instance left at the same time the first joined in two different ways:

1. I got an event on the Participant Collection OnUpdated which contained the large gallery participant instance as a removed participant, this event was logged by the bot. In the same way the joining of the new instance was logged, if the two events were in the same roster I can't tell 100% sure but the 2 logs do have the same timestamp in millisecond accuracy
2. When querying the participant collection for the media source ID of the first instance large gallery (the ID was cached by the bot and in video frames the bot received) the bot was not able to find the participant in the collection and the bot logged an Error message. (The query only ran after the participant removed event was received)

Thank you very much for your help, I really appreciate it. I already assumed that the large gallery works like this and further instances are there deliberately to help out the first instance 🙂

[ssulzer]

Hi @1fabi0 What version of the Graph Communications SDK are you using? This sounds like a bug in the SDK. Can you share any relevant SDK logs please? (e.g., around the time the new Large gallery bot participant joins and the SDK tells your app the original Large gallery instance has left) FYI @samarthasthana

[1fabi0]

Hi @ssulzer,

Here are the versions we’re using:

Microsoft.Graph.Communications.Calls: 1.2.0.10563 Microsoft.Graph.Communications.Calls.Media: 1.2.0.10563 Microsoft.Skype.Bots.Media: 1.28.0.401-preview

Regarding the logs, I've sent you an email as I prefer not to disclose certain IDs here.

Thank you very much!

Best, Fabian