clowdr
clowdr copied to clipboard
Live Transcription failing in video chats
- We repeatedly hit the "max 25 concurrent streams" connection limit (somehow - probably a side effect of the other issues below)
- A connection times out after 15 seconds if no audio data is received
- This possibly immediately triggers an attempt to reconnect if the person's mic is on even if they are still not making any noise. Likely to trigger rate limiting.
- The connection token is fetched when the camera is first switched on. But if their mic is switched on after, this won't necessarily cause a refetch of the token. The token expires if it isn't used within 5 minutes.
- The connection token isn't refetched if it expires or otherwise causes an error
- Connections needs to be created (and token possibly refetched) when the mic on/off status changes not just camera object change (the connection probably does recreate on mic toggle with the current code but the token won't be refetched)
- Connections should not be created unless the user is actually talking. If someone is sat silently (with mic on) then we shouldn't try to transcribe until they make noise (and we need to buffer that noise for pushing into the websocket once it is established)