twilio-video.js icon indicating copy to clipboard operation
twilio-video.js copied to clipboard

Published 20 hours ago • verified publisher icon twilio

client is unable to apply or create a local media description, windows chrome to windows chrome

Open cindyloo opened this issue 2 years ago • 21 comments

Consistently hitting this error on Windows 10 Chrome version Version 104.0.5112.81 when doing end to end testing between windows machines.

Expected behavior: Connecting at DeviceSelectionScreen should connect cleaning

Software versions:

  • [ ] Browser(s): windows chrome. (Works on edge)
  • [ ] Operating System:
  • [ ] twilio-video.js: 2.21.2
  • [ ]react

cindyloo avatar Aug 15 '22 15:08 cindyloo

bongo-upgrade.herokuapp.com-1660576497824.log

cindyloo avatar Aug 15 '22 15:08 cindyloo

android also we think

cindyloo avatar Aug 15 '22 15:08 cindyloo

Local Media Error.txt

Rcherry88 avatar Aug 15 '22 15:08 Rcherry88

and also in Mac OS Chrome:

<div aria-expanded="true" tabindex="-1" style="box-sizing: border-box; min-width: 0px; min-height: 0px; flex: 0 0 auto; color: rgb(48, 57, 66); font-family: menlo, monospace; font-size: 11px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(236, 241, 248); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><span class="source-code" style="box-sizing: border-box; min-width: 0px; min-height: 0px; font-family: var(--source-code-font-family); white-space: pre-wrap; font-size: var(--source-code-font-size) !important; line-height: 1.2;"><span class="console-message-anchor" style="box-sizing: border-box; min-width: 0px; min-height: 0px; float: right; text-align: right; max-width: 100%; margin-left: 4px;"><span class="devtools-link" role="link" tabindex="-1" title="webpack://_N_E/node_modules/twilio-video/es5/util/twilioerror.js?6b3f:54" style="box-sizing: border-box; min-width: 0px; min-height: 0px; color: var(--color-text-secondary); text-decoration: underline; cursor: pointer; word-break: break-all;"><br class="Apple-interchange-newline">twilioerror.js?6b3f:54</span> </span><span class="console-message-text" style="box-sizing: border-box; min-width: 0px; min-height: 0px; color: var(--override-error-text-color) !important;">Uncaught (in promise) <span class="object-value-error source-code" style="box-sizing: border-box; min-width: 0px; min-height: 0px; font-family: var(--source-code-font-family); white-space: pre-wrap; font-size: var(--source-code-font-size) !important; line-height: 1.2;"><span style="box-sizing: border-box; min-width: 0px; min-height: 0px;">TwilioError: Client is unable to create or apply a local media description
<span style="box-sizing: border-box; min-width: 0px; min-height: 0px;">    at MediaClientLocalDescFailedError.TwilioError [as constructor] (<span class="devtools-link" role="link" tabindex="-1" title="webpack://_N_E/node_modules/twilio-video/es5/util/twilioerror.js?6b3f:56:1" style="box-sizing: border-box; min-width: 0px; min-height: 0px; color: var(--color-text-secondary); text-decoration: underline; cursor: pointer; word-break: break-all;">twilioerror.js?6b3f:56:1</span>)
</span><span style="box-sizing: border-box; min-width: 0px; min-height: 0px;">    at new MediaClientLocalDescFailedError (<span class="devtools-link" role="link" tabindex="-1" title="webpack://_N_E/node_modules/twilio-video/es5/util/twilio-video-errors.js?25ab:871:1" style="box-sizing: border-box; min-width: 0px; min-height: 0px; color: var(--color-text-secondary); text-decoration: underline; cursor: pointer; word-break: break-all;">twilio-video-errors.js?25ab:871:1</span>)
</span><span style="box-sizing: border-box; min-width: 0px; min-height: 0px;">    at eval (<span class="devtools-link" role="link" tabindex="-1" title="webpack://_N_E/node_modules/twilio-video/es5/signaling/v2/peerconnection.js?7b1f:679:1" style="box-sizing: border-box; min-width: 0px; min-height: 0px; color: var(--color-text-secondary); text-decoration: underline; cursor: pointer; word-break: break-all;">peerconnection.js?7b1f:679:1</span>)
</span><span style="box-sizing: border-box; min-width: 0px; min-height: 0px;">    at async Promise.all (<span class="devtools-link" title="http://localhost:3000/index 0" role="link" tabindex="-1" style="box-sizing: border-box; min-width: 0px; min-height: 0px; color: var(--color-text-secondary); text-decoration: underline; cursor: pointer; word-break: break-all;">index 0</span>)
</span><span style="box-sizing: border-box; min-width: 0px; min-height: 0px;">    at async Promise.all (<span class="devtools-link" title="http://localhost:3000/index 0" role="link" tabindex="-1" style="box-sizing: border-box; min-width: 0px; min-height: 0px; color: var(--color-text-secondary); text-decoration: underline; cursor: pointer; word-break: break-all;">index 0</span>)</span></span></span></span></span></div><div class="" role="group" style="box-sizing: border-box; min-width: 0px; min-height: 0px; flex: 0 0 auto; color: rgb(48, 57, 66); font-family: menlo, monospace; font-size: 11px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(236, 241, 248); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial;"><span class="monospace" style="display: inline-block; --monospace-font-size:11px; --monospace-font-family:menlo,monospace; --source-code-font-size:11px; --source-code-font-family:menlo,monospace; box-sizing: border-box; min-width: 0px; min-height: 0px; font-family: var(--monospace-font-family); font-size: var(--monospace-font-size) !important;">

  | TwilioError | @ | twilioerror.js?6b3f:54
-- | -- | -- | --
  | MediaClientLocalDescFailedError | @ | twilio-video-errors.js?25ab:871
  | eval | @ | peerconnection.js?7b1f:679
  | Promise.then (async) |   |  
  | PeerConnectionManager.update | @ | peerconnectionmanager.js?9f43:372
  | RoomV2._update | @ | room.js?5fe1:392
  | emit | @ | events.js?8849:153
  | handleMessage | @ | twilioconnectiontransport.js?cacb:486
  | emit | @ | events.js?8849:153
  | TwilioConnection._handleMessage | @ | twilioconnection.js?ffae:454
  | eval | @ | twilioconnection.js?ffae:335

</span></div>

cindyloo avatar Aug 15 '22 15:08 cindyloo

We can also 99% reproduce this on mac Chrome/iOS safari

  1. Start new room & session on Chrome on my Mac at https://bongo-upgrade.herokuapp.com

  2. On my iPhone, connect to the same room via invite link on Safari.

  3. Mute both sides (this probably doesn't matter but I had to do it to not get feedback)

  4. Put iOS side to sleep (by pressing side button to put phone to sleep)

  5. I sometimes get the crash on Chrome on Mac

  6. If that doesn't happen, then wake up iPhone, disconnect the iPhone session, then repeat starting with step 2.

cindyloo avatar Aug 16 '22 18:08 cindyloo

debug log from twilio.connect:

og.js?a324:167 2022-08-16T19:31:16.766Z debug [RemoteAudioTrack #5: MT1a266a60bf1afd55a6918417ab71d3de] Started
log.js?a324:167 2022-08-16T19:31:16.771Z debug [PeerConnectionV2 #1: b09c895c-8acc-49b3-aea8-158197705b62] _updateEncodings: [0: true, 4], [1: true, 2], [2: true, 1]
log.js?a324:167 2022-08-16T19:31:16.773Z warn [connect #1] event {elapsedTime: 32760, group: 'media', level: 'warning', name: 'error', timestamp: 1660678276772, …}
Log.log @ log.js?a324:167
Log.warn @ log.js?a324:210
eval @ eventobserver.js?a0d2:82
emit @ events.js?8849:153
PeerConnectionV2._publishMediaWarning @ peerconnection.js?7b1f:754
eval @ peerconnection.js?7b1f:680
Promise.catch (async)
PeerConnectionV2._answer @ peerconnection.js?7b1f:678
PeerConnectionV2._updateDescription @ peerconnection.js?7b1f:1280
eval @ peerconnection.js?7b1f:1569
gotKey @ statemachine.js?aee9:157
Promise.then (async)
StateMachine.bracket @ statemachine.js?aee9:155
PeerConnectionV2.update @ peerconnection.js?7b1f:1559
eval @ peerconnectionmanager.js?9f43:370
eval @ peerconnectionmanager.js?9f43:365
Promise.then (async)
PeerConnectionManager.update @ peerconnectionmanager.js?9f43:364
RoomV2._update @ room.js?5fe1:403
emit @ events.js?8849:153
handleMessage @ twilioconnectiontransport.js?cacb:493
emit @ events.js?8849:153
TwilioConnection._handleMessage @ twilioconnection.js?ffae:454
eval @ twilioconnection.js?ffae:335
log.js?a324:167 2022-08-16T19:31:16.795Z debug [TwilioConnection #1: wss://global.vss.twilio.com/signaling] Outgoing: {"body":{"session":"a05357d8cc1c56807cd49502b3150eae26e72fae7146ed3a004b220bac1327dc0e18cb8555f043cc16546deeda40b866659b75a3816b0074416d716cee1f89ad","type":"update","version":2,"peer_connections":[{"description":{"type":"offer","revision":5,"sdp":"v=0\r\no=- 1204707875406607330 7 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1 application0 3 audio0 video0 4\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=audio 52940 UDP/TLS/RTP/SAVPF 111 0\r\nc=IN IP4 172.28.192.1\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate:3624685079 1 udp 2122260223 172.28.192.1 52940 typ host generation 0 network-id 2\r\na=candidate:1386685195 1 udp 2122194687 192.168.86.240 52941 typ host generation 0 network-id 1 network-cost 10\r\na=ice-ufrag:kEfz\r\na=ice-pwd:1reKL78X00NuDcDZoPd0l9Uz\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1D:EC:0A:3B:FF:FB:94:21:44:E5:2E:4F:B2:BF:DA:83:59:A1:A4:0C:14:FB:3C:66:4E:C9:50:F4:40:FC:AA:63\r\na=setup:actpass\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=sendrecv\r\na=msid:- dac95d64-bfd5-427f-af42-196e31ca0ed4\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=rtpmap:0 PCMU/8000\r\na=ssrc:3202717424 cname:MOrT/RJESsWZlE4/\r\na=ssrc:3202717424 msid:- dac95d64-bfd5-427f-af42-196e31ca0ed4\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 108 116 118 97 109 117\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:kEfz\r\na=ice-pwd:1reKL78X00NuDcDZoPd0l9Uz\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1D:EC:0A:3B:FF:FB:94:21:44:E5:2E:4F:B2:BF:DA:83:59:A1:A4:0C:14:FB:3C:66:4E:C9:50:F4:40:FC:AA:63\r\na=setup:actpass\r\na=mid:1\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:14 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:13 urn:3gpp:video-orientation\r\na=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=sendrecv\r\na=msid:- d824c6a5-3f0c-4509-bb93-5e11efee7d8c\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:108 H264/90000\r\na=rtcp-fb:108 goog-remb\r\na=rtcp-fb:108 transport-cc\r\na=rtcp-fb:108 ccm fir\r\na=rtcp-fb:108 nack\r\na=rtcp-fb:108 nack pli\r\na=fmtp:108 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:116 red/90000\r\na=rtpmap:118 ulpfec/90000\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:109 rtx/90000\r\na=fmtp:109 apt=108\r\na=rtpmap:117 rtx/90000\r\na=fmtp:117 apt=116\r\na=ssrc-group:FID 2399174641 2321330080\r\na=ssrc-group:FID 1049051876 1417694977\r\na=ssrc-group:FID 3942844383 3372620524\r\na=ssrc-group:SIM 2399174641 1049051876 3942844383\r\na=ssrc:2399174641 cname:MOrT/RJESsWZlE4/\r\na=ssrc:2399174641 msid:- d824c6a5-3f0c-4509-bb93-5e11efee7d8c\r\na=ssrc:2321330080 cname:MOrT/RJESsWZlE4/\r\na=ssrc:2321330080 msid:- d824c6a5-3f0c-4509-bb93-5e11efee7d8c\r\na=ssrc:1049051876 cname:MOrT/RJESsWZlE4/\r\na=ssrc:1049051876 msid:- d824c6a5-3f0c-4509-bb93-5e11efee7d8c\r\na=ssrc:1417694977 cname:MOrT/RJESsWZlE4/\r\na=ssrc:1417694977 msid:- d824c6a5-3f0c-4509-bb93-5e11efee7d8c\r\na=ssrc:3942844383 cname:MOrT/RJESsWZlE4/\r\na=ssrc:3942844383 msid:- d824c6a5-3f0c-4509-bb93-5e11efee7d8c\r\na=ssrc:3372620524 cname:MOrT/RJESsWZlE4/\r\na=ssrc:3372620524 msid:- d824c6a5-3f0c-4509-bb93-5e11efee7d8c\r\na=x-google-flag:conference\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:kEfz\r\na=ice-pwd:1reKL78X00NuDcDZoPd0l9Uz\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1D:EC:0A:3B:FF:FB:94:21:44:E5:2E:4F:B2:BF:DA:83:59:A1:A4:0C:14:FB:3C:66:4E:C9:50:F4:40:FC:AA:63\r\na=setup:actpass\r\na=mid:application0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 108 116 118 97 109 117\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:kEfz\r\na=ice-pwd:1reKL78X00NuDcDZoPd0l9Uz\r\na=ice-options:trickle\r\na=fingerprint:sha-256 1D:EC:0A:3B:FF:FB:94:21:44:E5:2E:4F:B2:BF:DA:83:59:A1:A4:0C:14:FB:3C:66:4E:C9:50:F4:40:FC:AA:63\r\na=setup:actpass\r\na=mid:3\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap
twilioerror.js?6b3f:54 Uncaught (in promise) TwilioError: Client is unable to create or apply a local media description
    at MediaClientLocalDescFailedError.TwilioError [as constructor] (twilioerror.js?6b3f:56:1)
    at new MediaClientLocalDescFailedError (twilio-video-errors.js?25ab:871:1)
    at eval (peerconnection.js?7b1f:679:1)
    at async Promise.all (index 0)
    at async Promise.all (index 0)
TwilioError @ twilioerror.js?6b3f:54
MediaClientLocalDescFailedError @ twilio-video-errors.js?25ab:871
eval @ peerconnection.js?7b1f:679
Promise.then (async)
PeerConnectionManager.update @ peerconnectionmanager.js?9f43:372
RoomV2._update @ room.js?5fe1:403
emit @ events.js?8849:153
handleMessage @ twilioconnectiontransport.js?cacb:493
emit @ events.js?8849:153
TwilioConnection._handleMessage @ twilioconnection.js?ffae:454
eval @ twilioconnection.js?ffae:335
client.js?374c:7 [Violation] Added non-passive event listener to a scroll-blocking 'touchstart' event. Consider marking event handler as 'passive' to make the page more responsive. See https://www.chromestatus.com/feature/5745543795965952
eval @ client.js?374c:7

cindyloo avatar Aug 16 '22 19:08 cindyloo

lastly here is a very helpful error in the console.log:

{elapsedTime: 25613, group: 'media', level: 'warning', name: 'error', timestamp: 1660755568010, …}
elapsedTime: 25613
group: "media"
level: "warning"
name: "error"
payload:
code: 53402
context: "{\"error\":\"Cannot read properties of undefined (reading 'maxActiveLayers')\",\"sdp\":\"v=0\\r\\no=- 3869744352 3869744352 IN IP4 0.0.0.0\\r\\ns=VMc2da30e29d382dc226f9e9d19bb89c15\\r\\nt=0 0\\r\\na=extmap-allow-mixed\\r\\na=msid-semantic: WMS\\r\\na=ice-lite\\r\\na=group:BUNDLE 0 1\\r\\na=msid-semantic:WMS *\\r\\nm=audio 17577 UDP/TLS/RTP/SAVPF 111 0\\r\\nc=IN IP4 3.235.111.145\\r\\na=setup:active\\r\\na=recvonly\\r\\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\\r\\na=mid:0\\r\\na=rtcp:17577 IN IP4 3.235.111.145\\r\\na=rtpmap:111 opus/48000/2\\r\\na=rtpmap:0 PCMU/8000\\r\\na=rtcp-fb:111 transport-cc\\r\\na=rtcp-mux\\r\\na=fmtp:111 minptime=10;useinbandfec=1;usedtx=1\\r\\na=ice-ufrag:kXpc\\r\\na=ice-pwd:4R6ScAqGmravGn6zXzkANW\\r\\na=fingerprint:sha-256 27:B3:D0:29:46:65:2B:8C:2D:45:16:AD:D8:D4:15:96:07:93:B5:54:37:D9:C0:FB:08:EE:83:C2:96:10:6C:2F\\r\\na=candidate:2 1 UDP 2013266430 3.235.111.145 17577 typ host\\r\\na=candidate:2 2 UDP 2013266429 3.235.111.145 19037 typ host\\r\\nm=video 17577 UDP/TLS/RTP/SAVPF 96 108 116 118\\r\\nc=IN IP4 3.235.111.145\\r\\na=rtcp-rsize\\r\\na=setup:active\\r\\na=recvonly\\r\\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\\r\\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\\r\\na=mid:1\\r\\na=rtcp:17577 IN IP4 3.235.111.145\\r\\na=rtpmap:96 VP8/90000\\r\\na=rtpmap:108 H264/90000\\r\\na=rtpmap:116 red/90000\\r\\na=rtpmap:118 ulpfec/90000\\r\\na=rtcp-fb:96 goog-remb\\r\\na=rtcp-fb:96 transport-cc\\r\\na=rtcp-fb:96 ccm fir\\r\\na=rtcp-fb:96 nack\\r\\na=rtcp-fb:96 nack pli\\r\\na=rtcp-fb:108 goog-remb\\r\\na=rtcp-fb:108 transport-cc\\r\\na=rtcp-fb:108 ccm fir\\r\\na=rtcp-fb:108 nack\\r\\na=rtcp-fb:108 nack pli\\r\\na=rtcp-mux\\r\\na=fmtp:108 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\\r\\na=x-google-flag:conference\\r\\na=ice-ufrag:kXpc\\r\\na=ice-pwd:4R6ScAqGmravGn6zXzkANW\\r\\na=fingerprint:sha-256 27:B3:D0:29:46:65:2B:8C:2D:45:16:AD:D8:D4:15:96:07:93:B5:54:37:D9:C0:FB:08:EE:83:C2:96:10:6C:2F\\r\\na=candidate:2 1 UDP 2013266430 3.235.111.145 17577 typ host\\r\\na=candidate:2 2 UDP 2013266429 3.235.111.145 19037 typ host\\r\\n\"}"
message: "Calling setRemoteDescription with an RTCSessionDescription of type \"answer\" failed"

cindyloo avatar Aug 17 '22 17:08 cindyloo

we had to avoid publishing the track on Windows 10 Chrome in this way:

const aCanvasTrack = getLocalVideoTrackFromCanvas(
          participant.sid,
          canvasOverlayBuff
        ).then((track: LocalVideoTrack) => {
          if (supportsCPublishing) { // windows 10 check
            (participant as LocalParticipant)
              .publishTrack(track)
              .then(localTrackPublication => {
                console.log(
                  `Track overlay was published with SID ${localTrackPublication.trackSid} for ${participant.sid}`
                );
              });
          }
        });

cindyloo avatar Aug 23 '22 15:08 cindyloo

@cindyloo ,

Thanks for writing in. Is this happening only for screen share tracks, or is it happening for all audio and video tracks?

manjeshbhargav avatar Aug 24 '22 18:08 manjeshbhargav

videoTracks @manjeshbhargav

cindyloo avatar Aug 26 '22 18:08 cindyloo

to reproduce: go to https://bongo-upgrade-testing.herokuapp.com begin the session as Participant A in windows 10/chrome browser, and send yourself an email (gmail works best) at the PreJoinScreen/RoomName screen. Click Continue. Click join then as Participant B, click the link from the email (if on same machine, use an incognito browser obvi) and follow the prompts similarly. Upon Participant B joining, You will see the error in Participant A's browser

cindyloo avatar Aug 30 '22 15:08 cindyloo

Hi @cindyloo ,

I tried to run the app, but I don't think it contains the changes I requested here. Can you make sure these changes are present in the deployed app?

manjeshbhargav avatar Sep 01 '22 16:09 manjeshbhargav

hi @manjeshbhargav- I just removed all processor code (sorry, I thought I had) and pushed. The canvas track (there's only one canvas video track per participant btw) publishes also

cindyloo avatar Sep 01 '22 16:09 cindyloo

@cindyloo ,

Thanks, I'll take a look.

manjeshbhargav avatar Sep 01 '22 18:09 manjeshbhargav

@cindyloo ,

I tried this URL: https://bongo-upgrade-testing.herokuapp.com/, and it still does not contain the changes I requested. Can you investigate?

manjeshbhargav avatar Sep 01 '22 23:09 manjeshbhargav

hi @manjeshbhargav the changes that you requested are indeed there: a) "where you do not add any video processor to the local video track"

  • all the code for processor is commented out within AnonymousVideoTrack.tsx b) "publish the (second video track which is the canvas track)? That way, it will be easier for me to debug."
  • the canvas track is published within AnonymousVideoTrack.tsx as long as there is a participant and a canvas element from which to publish.

what are you looking at that would make you think differently? Thank you

cindyloo avatar Sep 02 '22 12:09 cindyloo

@cindyloo ,

The app is still trying to publish the canvas video track with the mask effect. I was hoping the app just publishes the default camera track. Is that what you want to demonstrate? Anyway, it seems to be working for me on Mac (you said you saw the error on Mac too).

Screen Shot 2022-09-02 at 11 10 31 AM

I will ask my QE team to see if they can reproduce this on Windows.

manjeshbhargav avatar Sep 02 '22 18:09 manjeshbhargav

we more rarely get the Client error on mac (although it does happen). You need to try it on Windows 10 - it happens every time.

We are publishing the default camera track (both the Twilio default local and audio tracks as in useLocalTrack hook), but I don't get the error doing that. I get the error if I publish a new video track (from a canvas element) on Windows 10.

If you want, I could just publish additional track copying the video element track, but that wouldn't be the same scenario.... or publish an empty canvas track (vs one from media pipe)

ty

cindyloo avatar Sep 02 '22 21:09 cindyloo

Hey.

QE team was able to reproduce the issue on Windows, using the app provided above (https://bongo-upgrade-testing.herokuapp.com/). The steps are just the same that @cindyloo provided:

  • A joins the room using Bongo app in Windows machine (we validated using Win 11)
  • A sends an invite to the room to another user
  • B can use any OS (we reproduced the issue when B was using Mac)
  • B joins the same room
  • A receives error (Client is unable to create or apply a local media description)

However, we were not able to reproduce the issue in our demo apps. We tried to reproduce the similar scenario with our demo app, and we don't see an error:

  • Windows user creates one simple video track and second video track from CanvasCaptureMediaStreamTrack using captureStream() from canvas and joins the room
  • Another user joins the room with video track

cc @manjeshbhargav

DenissSazanovs avatar Sep 07 '22 07:09 DenissSazanovs

@DenissSazanovs did you publish the canvas track as well? that's what repeatedly caused the error for me and for #1857 apparently

cindyloo avatar Sep 07 '22 15:09 cindyloo

Hi @cindyloo ,

I may have a workaround for you while we fix the issue in the SDK. Please let me know if it works for you.

manjeshbhargav avatar Sep 22 '22 23:09 manjeshbhargav

Hi @cindyloo ,

I have released 2.24.2, which fixes the error you are seeing. I will close the issue now. Thank you for bringing this to my notice.

manjeshbhargav avatar Sep 29 '22 23:09 manjeshbhargav