DrBomb
Basic Linux support with OpenCV and QoL improvements
Hello!
I just found this, and I'm impressed with how good it is. This PR adds: basic Linux support using OpenCV, two new parameters to specify the camera index and capability index, and also fixes the mode switch, which did not work.
Hi! Thank you for considering making a PR to this little thing! Not gonna lie it is VERY basic, you're giving me more credit than I deserve haha. I've checked your additions and I think they look OK, I didn't use openCV as a backend as I saw it more fit to use DS as a way to get the "right" formats, but adding it for linux support is alright. Switches are OK, just I'm not expecting a lot of people to use them.
There are two things you might want to add to the PR if you want to call this linux compatible though.
- Not show RTX tracking or quit if chosen when ran on linux.
- Make sure requirements.txt is compatible with a linux install. Maybe create a different requirements file. PyGrabber is windows specific I think.
- Set up a "Linux Support" section in case installing opencv is non-trivial. I'm not familiar with that install.
Again, thank you for taking your time to add to the repo. I'm unable to test your PR right now as I don't have a linux machine, but I'm getting a new laptop next week so hopefully I will be able to test it then.
I didn't use openCV as a backend as I saw it more fit to use DS as a way to get the "right" formats
Camera support on Linux is what it is and there is no standard way to query camera information and OpenCV might as well just ignore whatever we tell it to do, I already used it for a stream and I did just fine so it should be ok as is.
Not show RTX tracking or quit if chosen when ran on linux.
It should be hidden now.
Make sure requirements.txt is compatible with a linux install. Maybe create a different requirements file. PyGrabber is windows specific I think.
Done, it installs only what it needs on each platform.
Set up a "Linux Support" section in case installing opencv is non-trivial. I'm not familiar with that install.
Nothing specific in this case, the lib includes everything for it to work.
Added a new change that replaces time.time() with time.perf_counter() as the former does not have enough precision and causes random crashes on Linux along with a small change to the semaphore to actually wait for the result and then send the data.
I'm not interested on changing the behavior of the semaphore. Mediapipe ignores frames while processing so it makes sense to keep feeding frames. The change from time.time to time.perf_time seem arbitrary as I'm not sure why you're having the crash. We're talking about potentially a 60fps webcam stream, what precision is needed that qualifies for perf_counter?
Also, please keep changes to a single PR. I need you to either roll back to the agreed changes or the PR will be rejected. We can discuss further changes once I get around testing this one.
Sure, I can back out the semaphore change.
Now, regarding time.time, it is not guaranteed to be more precise than a second and can be affected by the system clock. I experienced multiple crashes because mediapipe expects an increasing timestamp which sometimes did not happen hence the change.
I might add, at least on my systems, mediapipe is faster than the camera, not waiting for the semaphore means the output data will lag, at best, by one camera frame. By waiting we guarantee that it will always send the latest data since we are not waiting for the next camera frame to be read. We can discuss this in more detail in another PR.
I'm curious if this still has plans to be merged? Seems like this has been open for over a year and the changes never got merged but it could be extremely useful since with the current issues plaguing Win 11 I plan to jump ship and move to Linux soon
Honestly. I haven't touched this in awhile. Stuff like the ifs in the middle of the mediapipe setup are too messy for me to merge with a good conscience. You should be able to clone their branch and test it out on linux if you'd like.
This project was never written to support linux, but I get the need for it. I might revisit it soon as I'm trying to write a better abstraction layer so I can use mediapipe on VRChat.
Here's the first step for proper linux mediapipe support. https://github.com/DrBomb/MediapipeFaceLandmarker I will integrate it with the ExpressionAppBridge in the following days. Haven't tested it in Ubuntu though, need to set up my laptop again. Hopefully for next week it'll be integrated into this.
Here's the first step for proper linux mediapipe support. https://github.com/DrBomb/MediapipeFaceLandmarker I will integrate it with the ExpressionAppBridge in the following days. Haven't tested it in Ubuntu though, need to set up my laptop again. Hopefully for next week it'll be integrated into this.
Nice I look forward to it, also since we've had quite a few updates for Nvidia and stuff do we know if RTX tracking also works now as well? I would imagine it very well might if run under Proton or Lutris if ExpressionAppBridge on linux would be unable to launch it natively since Ray Tracing works on Linux
No, RTX tracking is from the binaries provided by VTube Studio so I will just skip offering that tracking when running from linux platforms. I won't be bothering to figure out any translation layers.
I guess the best way would be to compile the example RTX app to linux but I haven't started checking that out, the code is freely available though, perhaps it will work!