Magic-Sand
Magic-Sand copied to clipboard
thomwolf
Code upgrade to support Kinect V2
Hi Thom, I impressed by your project. I found the UC Sandbox project code impressive but very hard to customize due to the code structure. I therefore would like to test your approach. However, I would like to use the Kinect 2. As far as I can see , your project only support Kinect V1. My question is your estimate of the complexity of supporting Kinect 2. Probably it means adapting to freenect2. What is your estimate on the workload involved? Best, Rasmus
I think openframeworks supports Kinect V2 so it should be possible to use an openframworks kinect V2 add-on instead of the V1 that I use. I don't own a V2 so I have never looked deeply in the differences wrt V1. It is hard to estimate precisely the workload sorry, it depends on many things (your familiarity with C++ programming, the kinect API differences....) but my guess is that it is definitely possible to adapt the code to the V2.
Hi, Yes, I have indeed looked quite much into it. There are two issues. One is the cross-platform support for Kinect 2 that could probably be handled. The second is harder and is due to the fact that the Kinect 2 has a non-linear way of transforming "kinect coordinates" to "world coordinates" (caused by higher-order lens distortions). For the Kinect 1, this is done using a simple matrix multiplication and in Kinect 2 this is done using a pre-computed look-up table (that can be pulled from the firmware). Changing the Sandbox code to use Kinect 2 requires some re-coding of the coordinate transforms - all the way down to the shader code. I do believe I could do it - but am currently bogged down by teaching and grant application so the sandbox is hibernating in the lab. Finally, the Kinect 2 production has now sadly been stopped - so the motivation for putting a lot of effort into a dead-end product is limited. Best, Rasmus
Thanks for the thorough reply! Indeed the market of depth camera's is changing.
Well, I will try to look into this too - will keep you updated :)
Did it! -but Windows only, using ofxKinectforWindows2
I'll need some time to figure out how to share my efforts, just wanted to let you know it is possible :)
That's really great Sabrina! Don't hesitate to reach out if we can help, it would be great to have some support for Kinect V2! Did it involve a lot of modifications to the code ?
@SabrinaVerhage I'm also curious to hear about your progress. I'm trying to use the same Kinect 2 addon but am encountering some issues (very possibly related to multithreading). It would be great to see how you've managed to do this. Thanks!
I started by making a KinectV2Grabber based on the KinectGrabber class with some new functions to get world coordinates etc based on ofxKinectforWindows2 -so I kept the class threaded exactly like the original KinectGrabber. I also had to take the Kinect's resolution variable apart throughout the application because the Kinect V2's color & depth feeds have a different resolution, plus with the Kinect for Windows SDK the world's x & y axes are flipped (see https://msdn.microsoft.com/en-us/library/dn785530.aspx)
The calibration is working, the color map shaders are working; the average filter isn't working yet. I currently use a definitions.h that sets Kinect V2 or V1 with a bunch of ifdef's -so a bit messy :)
@thomwolf I'm not a Github Pro -how would you suggest I go about sharing this? Creating a forked version?
Yeah the different resolutions are definitely making things tricky. Have you tried using the IR frame instead of Color to keep everything in the same dimensions?
I think making a fork is the best and easiest way to share, because it creates your own copy but keeps a reference back to the original.
@SabrinaVerhage yes that would be the best way to do it. You can fork the project and push your modifications in the fork. Then, if it looks possible to combine it with the main branch, we can help you prepare a pull request to merge it with the main branch.
@SabrinaVerhage, now I am also trying to run this application with KinectV2 , but sadly couldn't get success . I am not much familiar with coding kinect , because this is my first project .Did you share your changes to make it work with kinect v2 ? It would be very helpful if we can get the changes.
Or any guidance on this topic would be grateful to run this application successfully.
我首先创建了一个基于 KinectGrabber 类的 KinectV2Grabber，其中包含一些新函数来获取基于 ofxKinectforWindows2 的世界坐标等 - 所以我保持该类的线程与原始 KinectGrabber 完全一样。我还必须在整个应用程序中分离 Kinect 的分辨率变量，因为 Kinect V2 的颜色和深度馈送具有不同的分辨率，加上 Kinect for Windows SDK，世界的 x 和 y 轴被翻转（请参阅https://msdn.microsoft .com/en-us/library/dn785530.aspx )
校准工作，颜色映射着色器工作；平均过滤器还没有工作。我目前使用的定义.h 将 Kinect V2 或 V1 设置为一堆 ifdef - 所以有点乱:)
@thomwolf我不是 Github 专家 - 你建议我如何分享这个？创建一个分叉版本？
I started by making a KinectV2Grabber based on the KinectGrabber class with some new functions to get world coordinates etc based on ofxKinectforWindows2 -so I kept the class threaded exactly like the original KinectGrabber. I also had to take the Kinect's resolution variable apart throughout the application because the Kinect V2's color & depth feeds have a different resolution, plus with the Kinect for Windows SDK the world's x & y axes are flipped (see https://msdn.microsoft.com/en-us/library/dn785530.aspx)
The calibration is working, the color map shaders are working; the average filter isn't working yet. I currently use a definitions.h that sets Kinect V2 or V1 with a bunch of ifdef's -so a bit messy :)
@thomwolf I'm not a Github Pro -how would you suggest I go about sharing this? Creating a forked version?
Is there any progress now? Currently I am also trying to make a Kinect 2 version
any progress on this V2 update? I also would like to try it if there's something new.
