kixelated
Issue publishing camera stream on the page
Greetings!
For some reason, I have issue publishing camera stream in moq-rs/web with the most recent changes in main-branch (without any modifications).
The notification light I have in my webcam blinks implying that the camera is shortly activated.
However, no video is shown on the page.
Chrome's console shows the following output:
rust_bg.js:1386 INFO moq-web/src/publish/video.rs:106 resolution = Dimensions { width: 640, height: 480 };
rust_bg.js:1386 INFO moq-web/src/publish/video.rs:39 config: VideoDecoderConfig { codec: "av01.0.04M.08", resolution: Some(Dimensions { width: 640, height: 480 }), display: Some(Dimensions { width: 640, height: 480 }), color_space: Some(VideoColorSpaceConfig { inner: VideoColorSpaceInit { obj: Object { obj: JsValue(Object({"fullRange":false,"matrix":"smpte170m","primaries":"smpte170m","transfer":"smpte170m"})) } } }), description: None, hardware_acceleration: None, latency_optimized: None }
rust_bg.js:1386 INFO moq-karp/src/broadcast.rs:259 loading catalog path = "demo/me/1743952810851/catalog.json";
rust_bg.js:1386 INFO moq-transfork/src/subscriber.rs:174 done track: "demo/me/1743952797142/catalog.json" id: 0
rust_bg.js:1386 INFO moq-transfork/src/subscriber.rs:155 active info = Info { priority: -1, order: Desc, latest: 0 }; track: "demo/me/1743952810851/catalog.json" id: 3
rust_bg.js:1386 INFO moq-transfork/src/publisher.rs:139 active info = Info { priority: -1, order: Desc, latest: 0 }; id: 0 track: "demo/me/1743952810851/catalog.json"
rust_bg.js:1386 INFO moq-web/src/watch/backend.rs:94 No video track found
None of the above is errors, just information implying that there's no video stream. Tried using another host (and another camera), but I'm getting similar output to browser's console.
The following is from the terminal (when running just all):
[srv] 2025-04-06T15:52:56.698852Z INFO session{id=9}: moq_relay::origins: announced origin path="demo/me/1743954776689/catalog.json"
[srv] 2025-04-06T15:52:56.717732Z INFO session{id=10}:publishing{track="demo/me/1743954776689/catalog.json" id=2}: moq_transfork::publisher: active info=Info { priority: -1, order: Desc, latest: 0 }
[srv] 2025-04-06T15:52:56.731700Z INFO subscribe{id=0 track="demo/me/1743954776689/catalog.json"}: moq_transfork::subscriber: active info=Info { priority: -1, order: Desc, latest: 0 }
[srv] 2025-04-06T15:53:07.877800Z WARN session{id=7}: moq_transfork::subscriber: announced error err=WebTransport(Session(ConnectionError(TimedOut)))
[srv] 2025-04-06T15:53:07.878378Z WARN session{id=7}: moq_transfork::session: terminated err=WebTransport(Session(ConnectionError(TimedOut)))```
Does anyone else have issues with publishing camera stream with the newest commits?
Or is the issue at my end only?
I tried to dig into this issue, and I think I've identified where this issue might be happening. P.S. My understanding of Rust is rudimentary, this is my first time reading/writing Rust code.
TL;DR
I believe the self.video is being set to None in the following code-block every time before we call broadcast.publish_video().
https://github.com/kixelated/moq-rs/blob/76f51770b59efc922782a3502732977237c73518/moq-web/src/publish/backend.rs#L51-L65
Commenting out self.video = None fixes the catalog, but runs into the following error:
Failed to execute 'decode' on 'VideoDecoder': A key frame is required after configure() or flush().
Findings
After logging the catalog, I noticed that the catalog.video was never being set. I suspected that the issue might be somewhere in the the web publisher module. I added logs in 2 places in moq-web/src/publish/backend.rs to see if broadcast.publish_video was ever being called.
I added log lines to log once self.video was being set correctly.
media = self.controls.media.next() => {
let media = media.ok_or(Error::Closed)?;
// Close the existing video stream.
self.video.take();
self.video_track.take();
if let Some(media) = media {
if let Some(track) = media.get_video_tracks().iter().next() {
let track: web_sys::MediaStreamTrack = track.unchecked_into();
// TODO: Perform this async if the delay is noticeable.
let video = Video::new(track).await?;
if let Some(broadcast) = self.broadcast.as_mut() {
let track = broadcast.publish_video(video.info().clone()).expect("failed to publish video");
self.video_track = Some(track);
}
tracing::info!(?video.info, "setting self.video");
self.video = Some(video);
tracing::info!("self.video was set successfully");
}
} else {
self.status.connection.set(ConnectionStatus::Connected);
}
},
I also added a log line to check if self.video was being read correctly in the tokio::select! block.
Some(session) = async { Some(self.connect.as_mut()?.established().await) } => {
let path = self.connect.take().unwrap().path;
let mut broadcast = moq_karp::BroadcastProducer::new(path)?;
broadcast.add_session(session?).expect("failed to add session to broadcast");
match &self.video {
Some(video) => {
let track = broadcast.publish_video(video.info().clone()).expect("failed to publish video");
self.video_track = Some(track);
}
None => {
tracing::error!("self.video was set to None");
},
}
self.broadcast = Some(broadcast);
self.status.connection.set(ConnectionStatus::Connected);
},
I also logged the URL in the first tokio::select! block (linked below).
https://github.com/kixelated/moq-rs/blob/76f51770b59efc922782a3502732977237c73518/moq-web/src/publish/backend.rs#L51-L53
Upon making these changes, I see the following logs:
INFO moq-web/src/publish/backend.rs:107 self.video was set successfully
rust_bg.js:1386 INFO moq-web/src/publish/backend.rs:52 got url url = Some(Some(Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(4443), path: "/demo/me", query: None, fragment: None }));
rust_bg.js:1386 ERROR moq-web/src/publish/backend.rs:79 self.video was set to None
My understanding of tokio::select! is not very clear, but it seems to me that self.video = None is running before we read self.video.as_mut() in the line below, causing it to never call broadcast.publish_video().
https://github.com/kixelated/moq-rs/blob/76f51770b59efc922782a3502732977237c73518/moq-web/src/publish/backend.rs#L71-L74
I'm not aware of a correct solution to this problem, I don't have much context about how this code works at the moment. If @kixelated can suggest a solution I'm happy to try to implement it. I'll also try to hack around the code and see if I can make it work when I get some free time.
Hey @roerohan, I've got a pretty large set of changes in the works including a revamp of publishing. It might be best to hold off until then unless you need something working now.
Alright, sounds good! I don't need it working urgently, I was just playing around with it and noticed this issue. Thanks for letting me know.
Okay publishing is now in Javascript and it's much better. I'm using it extensively for my application so it won't accidentally break again (I hope).