ProjectNeura
Remote Live Onboard Camera Feed from LEADS
Description It has always been the intention to include on board cameras on the vehicle and although not being used directly by LEADS a remote "live view around the vehicle" for the pit would be very useful.
Solution Since LEADS runs on Linux, a suggested solution is to merge the onboard camera feeds into a single virtual camera using OBS Studio and streaming the the live video feed using MS-Teams for Linux on the rpi4.
Alternatives None considered at the moment.
Additional context To be updated.
I have assigned @fz20181223luogu the task of becoming familiar with the virtual camera setup with OBS Studio and test the idea on a Windows machine running Teams for now. @ATATC maybe we can get him familiarized with then implementing on a rpi4 - would purchasing a second rpi4 make sense so we can run tasks in parallel? We will have 2 vehicles eventually so a second unit would not go to waste. Lmk.
Definitely. That would increase our efficiency.
@ATATC Can you please confirm that I am looking for a SC0195(9) - Raspberry Pi 4 8GB Rev 9? I know you made your purchase from Sayal for $149.95 but I am seeing it from DigiKey for $107.40 just need to make sure they are the same version to minimize software config issues.
If you come by rm217 or the workshop tomorrow, I'll have 2 usb cameras that you can try. The 2nd rpi4 has also been shipped so I expect by Wednesday you should also have a rpi4 to configure and then use to test the camera setup. We may also want to consider a 3-camera setup: FV, LRV and RRV. How easy is this to setup?
I think it is as easy as adding another source. Since two sources have been added to the virtual camera, I only need to follow previous steps that I have done to add another source.
Perfect! I can also confirm that we will have the 2nd rpi4 for you to configure and test your setup for tomorrow.
Installation scripts are here: /scripts/obs-install.sh and /scripts/obs-run.sh
Currently Mr. Mascarenhas recommended me a layout which front view is placed at the top, and two rear views at the bottom. Since the ratio of resolution about cameras I found today was 4:3 (maybe further config is needed as the website noted the maximum resolution is 1920 x 1080 which is a ratio of 16:9), if using that layout the ratio for output would be 8:9 (without stretching) which is looking weird (at least for me). Plus, I cannot present it fully in teams (probably because teams restricted ratio for camera to be a ratio close to 16:9).
Now I am thinking of another layout, which front view is placed on the left, and two rear views on right side, with left one above right one. If we use that layout, then the ratio for output would be 2:1 (without stretching) which is pretty close to 16:9 and being convenient to share. (At least more information on virtual camera can be kept)
The following screenshot shows how a virtual camera output in a ratio of 8:9 would look like in Teams:
The following diagram shows two virtual camera layouts from Mascarenhas and me (Scale is not guaranteed):
Let's test a few format options in some Teams calls before we settle on a format. Your suggestion @fz20181223luogu is fine - it's the immediate viewer perception that we're trying to accommodate. ps: having remote connectivity to the vehicle opens up the possibility for us to switch to different layouts (I think) but let's keep that on ice for now and keep this simple.
After today's test I believe that my layout suggestion works well. Here are some todos I would suggest now:
- Every time when cameras are reconnected to computer (including restarting computer), OBS cannot pair the video capture device source with the right camera, which need me to manually pair cameras every first time launching OBS after cameras are reconnected.
- My hypothesis on this problem is because all 3 cameras are named "USB Live Camera" which confuses OBS to pair the cameras, as all 3 cameras are produced by Angetubes and they are under a same model. Changing the names of cameras to be distinct names or using different models of cameras (not recommended since increasing in cost) might be the solution.
- The resolution of three cameras are too small (specifically, 640 x 480 px for all of them) which I would believe that 1280 x 1080 px for front view and 640 x 540 px for two rear views could be better under my paradigm of layout.
Thankfully, the disorder of cameras can be resolved by using a USB hub with all cameras plugged in. The problem with low resolution on three cameras is also solved by proper configuration in OBS.
This is great news @fz20181223luogu - nice work! Any headway on what the documented current draw of the 3 cameras is? @ATATC I think we should use the powered usb hub for the cameras to reduce the rpi power demand and the non-powered hub for the sensors since those devices are drawing power from the power controller.
This is great news @fz20181223luogu - nice work! Any headway on what the documented current draw of the 3 cameras is? @ATATC I think we should use the powered usb hub for the cameras to reduce the rpi power demand and the non-powered hub for the sensors since those devices are drawing power from the power controller.
Right. That's what I suggested.
That's what I thought @ATATC until I saw you were using the powered hub for the sensors yesterday. @fz20181223luogu you will need to switch hubs for the cameras - we can test this out next week.
The ANKER hub is the non-power hub. The other hub (that's in the shoebox) is the power hub via the USB-C port.
I bought 2 hubs. The Aceele 5-port hub that you are using in the box is the powered hub: "Smart Power Type-C Port: Each USB 3.0 port of the USB splitter can supply up to 900mA of current to power your USB devices, and it also has a Type-C port (5V/3A) for powering the hub. (The Type-C port can only power the hub, and doesn’t have other functions such as data transmission)" The ANKER 4-port hub that Jerry has been using is the non-powered hub.
@ATATC @fz20181223luogu we need to test with a MS-Teams call and OBS setup on the rpi. Can we set a date next week to do this please?
I won't be at school next Tuesday and Wednesday due to the AP exam. I need to see Ms. Rivera and Mr. Poe at some point. I guess we can try Monday at 2:30 and quickly finish it.
@fz20181223luogu Completed test today - MS-Teams runs fine and can switch between the 3 cameras on on the hub. We now need to figure out what the issue is with OBS on the rpi.