linux-v4l2: Save device by id or path

[ggodlewski]

Description

Use /dev/v4l/by-id or /dev/v4l/by-path when available in order to persist device binding to specific source. Fixes #3003 Closes #3437

Motivation and Context

Currently v4l2 uses /dev/videoX which is not consistent across reboots and device reconnections.

This patch uses directories:

• /dev/v4l/by-id - User has 2 devices: C920 and microscope. Sometimes I use them with another computer. I want to keep them assigned to specific sources/scenes without remembering which USB port to use.
• /dev/v4l/by-path - User has 2 devices: front camera connected to first usb and top camera connected to second usb. If one of the devices is upgraded/changed it remains assigned to the same path.
• /dev/videoX - Devices which are not present in above directories (eg. loopback)

The device can exists twice on the list (by-id vs by-path addressing). It's up to user which connect to the source.

In case of by-path device the source name consist of "bus_info (device name)" Otherwise it uses current naming: "device name (bus_info)"

Fixes #3003 Closes #3437

How Has This Been Tested?

Tested with 5 different v4l2 devices:

ID 04fc:6353 Sunplus Technology Co., Ltd General USB Device (Lamax Electra) ID 0c45:6366 Microdia Webcam Vitade AF (Digital microscope) ID 534d:2109 MacroSilicon USB3.0 HD VIDEO (Video grabber) ID 046d:082d Logitech, Inc. HD Pro Webcam C920 ID 046d:085e Logitech, Inc. BRIO Ultra HD Webcam

Test Case 1:

• Plug all the devices, assign them by id, put them on the scene
• It should display all the feeds

Test Case 2:

• Having all devices assigned by id switch their USB ports.
• The scene shouldn't change, all feeds in their old places.

Test Case 3:

• Having all devices assigned by id turn off OBS
• Run OBS
• Nothing is on the scene
• Plug devices in another order
• Devices should be assigned to correct places by udev

Test Case 4:

• Plug 2 devices, assign them by path, put them on the scene
• Switch USB ports
• The scene should have switched cameras

Test Case 5:

• Create v4l2loopback: sudo modprobe v4l2loopback devices=1 video_nr=88 ; ffmpeg -re -f lavfi -i "movie=something.mp4" -f v4l2 /dev/video88
• The v4l2loopback should be available on the list and working

Types of changes

• Tweak (non-breaking change to improve existing functionality)

Checklist:

• [x] My code has been run through clang-format.
• [x] I have read the contributing document.
• [x] My code is not on the master branch.
• [x] The code has been tested.
• [x] All commit messages are properly formatted and commits squashed where appropriate.
• [x] I have included updates to all appropriate documentation.

[RytoEX]

If this fixes/closes issues or other PRs, use "Fixes" or "Closes" before the Issue/PR reference to automatically link them together.

Does this resolve the lingering concern raised in #3437?

it seems like it v4l2loopback devices dont end up registered under the /dev/v4l/ tree. So there is probably more work to also include those video devices without paths.

[RytoEX]

I don't think GitHub picks up the keywords if you use a colon between them and the reference.

See:

• https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue
• https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/using-keywords-in-issues-and-pull-requests

[tytan652]

Does this resolve the lingering concern raised in #3437?

From PR description:

• /dev/videoX - Devices which are not present in above directories (eg. loopback)

Tested with two cameras and v4l2loopback devices.

Seems like a yes.

[kkartaltepe]

Im not a big fan of having both id and path options since I don't think there is really a good way to expose this to the user. I dont think that anyone will get the intention behind the two names in the list without reading this PR. I was leaning towards saving by path but checking with windows it looks like we save by ID there so id prefer we just keep it consistent with windows and only expose by-id to users.

I like the code structure though, seems like a good solution to the issue of virtual devices.

[ggodlewski]

As a user I'm unable to remember which USB port I used previously for specific camera. Linux exposes 2 ways to address v4l2 device to give users some flexibility. I think it's a good idea to expose this choice to OBS users as well. I added prefix: 'Bus/Device' to unique_device_name.

[tytan652]

Between /dev/v4l/by-id and /dev/v4l/by-path, is there one that manage correctly the scenario where the user connect two device from same model without getting them swapped between reboot.

/dev/v4l/by-path seems to be the only one that could avoid this.

Linux exposes 2 ways to address v4l2 device to give users some flexibility. I think it's a good idea to expose this choice to OBS users as well.

When you want to give choice, you need to check if any user that knows nothing about how device works on Linux could understand the choice they had to/can make.

Here you put everything in a list without even telling the difference between them.

[kkartaltepe]

I pushed some changes to clean up the freebsd defs. I think the code is fine now but I still dont believe the UX is really usable and we must choose one or the other of id or path. Or find a better UX.

[tytan652]

Maybe with #5160 we could add information with another property list which makes the user choose between /dev/v4l/by-path (as default) and /dev/v4l/by-id. And also a third option for /dev/videoX .

• Connection path (/dev/v4l/by-path)
• Device ID (/dev/v4l/by-id)
  • Warning: If you use same model devices, they could get swapped between reboots
• "My device does not show" (/dev/videox)
  • Warning: This way of selecting devices may not be consistent between reboots

[ggodlewski]

Another property list is more clicking. Most users have just one camera. #5160 is still open. #3437 was created almost 2 years ago. I don't think we should make this PR dependent on it.

Another, simple proposition: Lets add disabled item between groups saying: "Advanced. Select device by connection:"

Normal user won't bother to click this list. User with 2 cameras will select by id and ignore rest of the list. User with cameras of the same model will use the "advanced-by-path" list.

[koo5]

Thanks, this patch is a lifesaver.

I would suggest that attempts to make this user-friendly by dumbing things down and hiding them and coming up with subjective labellings like "Advanced:..." and "Connection path" ... actually complicates it for 95% of users. And it's not a "Danger" if your cameras show up the other way around. No credentials are gonna get leaked. You just notice it in the process of tweaking your scenes. And this is linux. We have all devices in a filesystem tree, we are proud of it, and we are not afraid to use it. I'd prefer the dropdown to look basically like what you get when you run: ls /dev/video* /dev/v4l/by-id/ /dev/v4l/by-path/, just with the "friendly" names appended to each item (even these friendly names are only friendly as long as you only have one device of the same "friendly" name, beyond that, they are an attack on your sanity.

/dev/v4l/by-id:
    usb-MACROSILICON_USB_Video-video-index0@  (Bla bla)
    usb-MACROSILICON_USB_Video-video-index1@  (Bla bla)
/dev/v4l/by-path:
    pci-0000:06:00.1-usb-0:2:1.0-video-index0@ (Bla bla)
    pci-0000:06:00.1-usb-0:3:1.0-video-index1@ (Bla bla)
/dev/video* (Warning: This way of selecting devices may not be consistent between reboots):
    /dev/video0 (Bla bla)
    /dev/video1 (Bla bla)

[ggodlewski]

I've done more testing and resolved one bug. There was a problem because UDEV is always using real path to device instead of by-* scheme. Now you're able to reconnect camera to different USB port during streaming or even start without any cameras and plug them during work. I'd appreciate some testing.

[tytan652]

Now that #5160 is merged, I reiterate my proposal:

We could add information with another property list which makes the user choose between /dev/v4l/by-path (as default) and /dev/v4l/by-id. And also a third option for /dev/videoX .

• Per connection (e.g. USB port), /dev/v4l/by-path
• Per device model, /dev/v4l/by-id
  • With a warning: If you use same model devices, they could get swapped between reboots
• "My device does not show", /dev/videox
  • With a warning: This way of selecting devices may not be consistent between reboots

Why /dev/v4l/by-path is the default ? Because between /dev/v4l/by-id and /dev/v4l/by-path, the latter is the one that manage correctly the scenario where the user connect two devices from same model without getting them swapped between reboot considering that the user rarely move his set up (so rarely change connection port).

Another property list is more clicking. Most users have just one camera.

Adding potential clicks doesn't mean bad UX because it can be avoided with a good default (here with /dev/v4l/by-path) with descriptions or tool-tips if necessary.

[ggodlewski]

@tytan652 it this case someone needs to fork this PR and prepare working proposition. I already spent more time than expected. My goal was to provide smallest changeset to solve unpredictable OBS behavior while selecting cameras. Maybe this PR can be merged (because it's better than current state) and UI improvements can be made as a separate PR? #3003 / #3437 are open for about 2 years.

[christephan1]

001-v4l2-hotplug.patch.txt

I have a working patch for your reference. Tested on Ubuntu 22.04. It supports selecting v4l2 camera by ID. Also hot-plugging USB from 1 port to another. But my work is based on the Ubuntu 22.04 LTS version of obs-studio-27.2.3+dfsg1-1

What I did is below:

1. Apply the draft in #3437
2. We use by-id instead of by-path. So changed to by-id.
3. Actually no need to readlink, those are symlink back to for example /dev/video0. If we readlink, then it doesn't work as expected.
4. Fix the udev. I assume udev will report the /dev/video0 name in "added" or "removed" callback. So, when comparing the hot-plug device with device_id, we need realpath() of device_id, because our device_id is symlink.
5. I don't assume the device_removed() callback is called before the by-id symlink disappear. So, in v4l2_init(), save the realpath() of device_id to v4l2_data for used in device_removed()
6. Fix The CMakeLists.txt udev issue. (Looks this already been fixed in master branch.)

[ggodlewski]

@kkartaltepe can you review it again? I've bought more cameras and tested it again (described in How Has This Been Tested?). Some memory leaks were removed. Also solved some udev problem. The UI creation was extracted into separated function: v4l2_build_device_list_ui if anyone wants to improve it in the future, it is trivial.

[kkartaltepe]

Sure ill take another look, but it will probably be a week or two.

[DarkDefender]

@kkartaltepe do you have time to take a look at this again in a few weeks? I don't know if @ggodlewski is still around though. I'm just asking as I'm eager to have this merged upstream (less third party patches to apply to my own builds).

[ggodlewski]

@DarkDefender I will not provide any updates to this project. I wasted many hours to prepare my patches and I bought 2 more devices to do a proper testing. Maintainers ignored me so someone else has to work on that. Obviously linux support is not a priority.

[DarkDefender]

I don't know if you had some additional communication with the developers, but I wouldn't attribute any of this to malice or unwillingness to make linux support better.

As I am a Blender developer myself, I do see the same thing on our side of things regarding code contributions. We get a lot of pull requests but we don't really have time to properly spent the time needed on all of them. Usually that means that contributors has to jump through hoops to get things merged. It is not a good situation, but I there is simply not enough developers around to properly take care of every contributor.

I kinda see the same thing here. @RytoEX do you have any time to get back to reviewing this? If not, is there anyone else we can collaborate with?

[RytoEX]

@RytoEX do you have any time to get back to reviewing this? If not, is there anyone else we can collaborate with?

I am not a Linux-based dev. My previous reviews here were only cosmetic. The only Linux-based contributor that I can think of who I would ask to review this is @kkartaltepe , who volunteers their free time as available. I can't force them to have more free time or force them to look at this.

[DarkDefender]

@RytoEX would you be fine if I step in as a reviewer and test this so I can vouch for the changes? We do have a lot of webcams and capture cards at the Blender Studio, so I can quite easily create a very complex setup to stress test it.

I'm sure kkartaltepe didn't intentionally abandon this patch as he had a patch of his own addressing the same issue. (But his patch was seemingly abandoned in favor of this one). I think it is just that he legitimately doesn't or didn't have time.

So if I am allowed to step up and help this get merged, I will happily do so. Especially because I'm going to include this patch in the Blender Studio OBS setup either way. However, I would like others to be able to use multiple v4l sources without ordering issues on reboot.

[kkartaltepe]

I did look at it again and it again seemed fine from a code perspective. But as I said " I think the code is fine now but I still dont believe the UX is really usable and we must choose one or the other of id or path. Or find a better UX." and as the author said "@tytan652 it this case someone needs to fork this PR and prepare working proposition. I already spent more time than expected."

This is reasonable code and probably will help whoever wants to do that fork and prepare something with better ux, but for me im not really convinced we should merge this as is and then try to change the UI again later. In fact merging this will probably just prevent us from ever making it more user friendly as such changes would probably break users who depend on having all these options available.

Its definitely at least partially my fault if I wasnt clear the main problem here is not the code but the UX, and at least my position is it should just work like windows.

[DarkDefender]

Thanks for clarifying your position

Main problem here is not the code but the UX, and at least my position is it should just work like windows.

But the issue here is that under the hood this doesn't work like on Windows. So to me, trying to force the interface to be like Windows would be a mistake.

To me, it seems like these two options that tytan652 mentioned are very useful and should be exposed:

• Per connection (e.g. USB port), /dev/v4l/by-path
• Per device model, /dev/v4l/by-id

This is because in my experience having "Per connection" would be a life saver when running bigger events like the Blender Conference. There we had cameras or capture cards break during the event and being able to simply delegate certain usb ports to a specific video input would make swapping out equipment really easy. So instead of requiring people to change source settings, they could simply just plug in the replacement device in the same port and it will work without any further intervention.

For most users though, I think they will most likely just use the "Per device" mode. (IE, they can plug the device in where ever they want and it will be mapped to the same source no matter what)

To me simply having the option in the UX to select which method you want to use shouldn't be too confusing or hard to understand. So I don't see why we would need to restrict ourselves to one or the other.

That is how I envision it at least. Do you agree that it sounds reasonable? If not, perhaps we can discuss it some more to see if we can come to an agreement?

but for me im not really convinced we should merge this as is and then try to change the UI again later

100% agree. The design of how we want things to work should be agreed upon and implemented before merging. Otherwise it could just lead to things not getting done later.

[ggodlewski]

We do have a lot of webcams and capture cards at the Blender Studio

@DarkDefender do you have any of Logitech BRIO or Kiyo Pro cameras? There is also #7346 PR which gives more controls like FOV and HDR. I wasn't able to find anyone to test it.

As I am a Blender developer myself, I do see the same thing on our side of things regarding code contributions.

Oh, what a coincidence. I wanted to provide something for Blender and now I'm afraid ;).

[DarkDefender]

I'm unsure if we have a Brio model, I have to check. But I do know that we have a Logitech MeetUp camera. It seems like it at least might be able to use the tilt and pan controls. So I could try that out if you want to :)

Oh, what a coincidence. I wanted to provide something for Blender and now I'm afraid ;).

I think it is much less frustrating if you know what to expect before hand. Before writing any code, I can recommend that you try to contact the developer in charge of the area you want to work in and talk through what you want to contribute. This usually makes the process a lot smoother as then you can come to an agreement about:

1. Will you contribution be something that would be accepted in the first place
2. If so, how will it be implemented
3. Get a picture of when the developer in question has time to review and merge your code.

BTW, with the updates posted in this pull request, do you want to continue or should I try to take over? Of course I haven't heard back yet from kkartaltepe, but for me personally I just think we only need to polish the UI presentation a bit.

It should be nice and intuitive if we either do two buttons like the "Vertices/Edges" toggle buttons in the picture to change between "by-id" and "by-path": Then the user can hover over those buttons and we can have tool tips that explain them in detail. Of course we could also just have it be a drop down menu for the "mode selection" instead (with tool tips there as well of course).

This is so it is very clear which "mode" the v4l source is in. What do you think?

[ggodlewski]

I do know that we have a Logitech MeetUp camera. It seems like it at least might be able to use the tilt and pan controls. So I could try that out if you want to :) For me it's so useful to finally have proper FOV support under Linux.

That would be cool. You'll probably need to add GUID for this camera. I curious about the result.

BTW, with the updates posted in this pull request, do you want to continue or should I try to take over?

My job here is done. The last thing I can maybe do is to rebase #7346 after this is merged. Feel free to take it over.

The interface looks cool.

[DarkDefender]

@kkartaltepe do you think you will have any time to spend on discussing the UI design? As it seems like you are OK with the code, perhaps I could get someone else involved if you don't have time. Because the remainder of the potential changes do not really need any Linux specific knowledge or expertise.

[kkartaltepe]

@kkartaltepe do you think you will have any time to spend on discussing the UI design? As it seems like you are OK with the code, perhaps I could get someone else involved if you don't have time. Because the remainder of the potential changes do not really need any Linux specific knowledge or expertise.

the best place to have a design discussion is probably on the discord where most developers are. There may be other people more interested in supporting multiple card lookup strategies.

[DarkDefender]

That sounds like a great idea! I didn't think about hopping onto discord to do this. But it makes total sense :sweat_smile:

[keriszafir]

Dealing with this issue, got an additional problem with camera disconnect/reconnect not being detected if I change the "device_id":"/dev/video2" etc. in the scene collection json file. This will break automatic reconnection and feed restart.

Test case and observations:

1. Fresh OBS config, a scene with a single camera set in OBS, but two cameras present in the system: works okay and the video feed comes back after the camera is disconnected and reconnected. On OBS/system restart, the other camera feed may appear instead of the previously selected one.
2. To mitigate this and keep cameras persistent, I made custom rules for my cameras, adding symlinks in /dev directory, e.g. KERNEL=="video[0-9]*", ATTR{name}=="Razer Kiyo Pro", ENV{ID_V4L_CAPABILITIES}==":capture:", SYMLINK+="video-kiyo" The ENV{ID_V4L_CAPABILITIES}==":capture:" match ensures that only the device file with video capture rather than metadata capabilities will be symlinked.
3. Then I edited the ~/.config/obs-studio/basic/scenes/{scene_collection_name}.json while OBS was not running, changing the video capture device settings manually "device_id":"/dev/video2" to "device_id":"/dev/video-kiyo" or "device_id":"/dev/v4l/by-id/usb-Razer_Inc_Razer_Kiyo_Pro-video-index0". The new setting is persistent and no video input devices were swapped anymore, but it comes at a price. Both custom-edited device_id entries break the automatic (udev based) disconnection and reconnection detection. I started OBS from terminal observing the log. After pulling the camera's USB plug, the video feed freezes, but it doesn't restart after plugging it back in (which is the expected behavior and it was the case with "device_id":"/dev/video2"). I don't get info: v4l2-input: Device {path} disconnected and info: v4l2-input: Device {path} reonnected log entries respectively. In case of unstable USB connections like on my Kiyo Pro, this will lead to camera feed freezing and not restarting, forcing me to close and restart OBS.

Multiple cameras of the same model aren't handled gracefully by the udev persistence rule (in Debian bookworm/sid: /lib/udev/rules.d/60-persistent-v4l.rules):

ACTION=="remove", GOTO="persistent_v4l_end"
SUBSYSTEM!="video4linux", GOTO="persistent_v4l_end"
ENV{MAJOR}=="", GOTO="persistent_v4l_end"

IMPORT{program}="v4l_id $devnode"

SUBSYSTEMS=="usb", IMPORT{builtin}="usb_id"
KERNEL=="video*", ENV{ID_SERIAL}=="?*", SYMLINK+="v4l/by-id/$env{ID_BUS}-$env{ID_SERIAL}-video-index$attr{index}"

TEST!="index", GOTO="persistent_v4l_end"
ATTR{index}!="?*", GOTO="persistent_v4l_end"

IMPORT{builtin}="path_id"
ENV{ID_PATH}=="?*", KERNEL=="video*|vbi*", SYMLINK+="v4l/by-path/$env{ID_PATH}-video-index$attr{index}"
ENV{ID_PATH}=="?*", KERNEL=="audio*", SYMLINK+="v4l/by-path/$env{ID_PATH}-audio-index$attr{index}"

LABEL="persistent_v4l_end"

but this file could be overriden with a custom rule in /etc/udev/rules.d with more fine-grained rules e.g. adding serial number or other unique identifier from the device's attributes. There may be a way, it's just system-level settings rather than OBS.