added videocapture development branch

[dmedine]

I just pushed a little project that I worked on for a few days before it defeated me and I got distracted by other things. Basically, I am envisioning an LSL based videocapture app that works like VideoStream, but is OS independent and doesn't rely on the Embarcadero development products.

The app is designed to spawn n number of video recording panels. These each have some fields for defining recording parameters such as frame rate and such. As the file is being written, it sends an LSL stream that consists of integers that refer to each video frame that is being recorded. This is exactly how VideoStream works.

With Qt 5, it should be relatively straight forward to select camera devices, video containers, screen resolution, etc. and record a video file -- except on Windows for which these functions are not provided by Qt. Anticipating headaches, I dove straight in to the Windows version of this app and chose (perhaps incorrectly) Windows Media Foundation (WMF) as the backend library for the video stuff. I sort of got it working for some cameras, but not for the one I wanted. I never actually succeeded in getting the user-defined WMF video writing callback function working properly. The code that attempts to start the recording is wincapture.cpp and wincapture.h.

Perhaps someone out there with more knowledge of WMF could spot what I did wrong. I used the example MFCaptureToFile as a template. This is part of the Windows SDK and on my machine the project lives at C:\Program Files\Microsoft SDKs\Windows\v7.1\Samples\multimedia\mediafoundation\MFCaptureToFile. I think I am doing everything correctly, but I have heard tell that one can't make calls to certain WMF functions from any thread but the main thread of an application. This might be my problem, but I tend not to believe this since obviously there are Windows applications that are capable of recording video in the background. If you like torturing yourself, you can read about this on a stack overflow stub that I started a while back.

Eventually, this app should work with all 3 desktop OSs. For OSX and Linus, Qt 5 will be the way to go. That part should be relatively painless and hopefully I will find the time to do it (unless someone else wants to...). But, I'm stuck on the Windows side of it. Maybe I should have used DirectShow all along, but I read that Windows is phasing that out in favor of WMF.

[cboulay]

I don't know if this will be useful to you, but I can offer my related experience.

In PSMoveService we do optical tracking of a coloured sphere in a scene using one or more cameras. Because we're doing some computer vision to get the sphere contour (and then some algebra to get its 3D position), and we're using OpenCV anyway, I just wrap OpenCV's interface to capture devices. There are better tutorials on how to use OpenCV to capture video, but here is a basic example of how we use it. It iterates through all compatible cameras that it finds on the system. In Windows, it uses DirectShow as a backend, and that picks up just about anything.

Because the camera we are using the most (the PS Eye camera) is not supported natively in Windows nor MacOS, I had to write an interface for custom 'drivers' (3 of them!) here. I doubt you'll have to do that unless you intend to use anything that isn't supported by DirectShow. We actually commented out our fallback to any default OpenCV camera because it was picking up and opening cameras from VR systems. We'll switch over to a blacklist soon, but for now just ignore that part.

OpenCV is actually terribly slow at displaying video frames (maxes out at around 30 fps), but it's perfectly good at capturing them and doing math on them. But you can take the data and display it in an SDL2 frame or a Qt widget, either of which is much more performant.