Future of Azure Kinect Component?

[xiangzhi]

I just found out that the AzureKinect team released a new version of their body tracking API (v1.1). They mention in their changelog that they won't be releasing a NuGet version of the code.

[Breaking Change] No NuGet install.

Since the AzureKinect component relies on the NuGet version of arbt, this likely means we will be stuck at 1.0.1

I was wondering if this is something y'all are aware of and might have a non-NuGet version released in the future? If not, I might take a stab at writing one since my study depends on it.

[chitsaw]

Thanks for letting us know. As of right now, we do not yet have a plan on how to evolve the AzureKinect component. We're following up with the body tracking team to find out more. Are you planning to do this work on Windows or Linux?

[xiangzhi]

No problem! I probably will try to do this on Windows. I have other stuff on my plate right now but I would let y'all know if I make any progress.

[xiangzhi]

I took at stab at the problem and made some progress. The changes turn out to quite minor. Here's a branch with the https://github.com/CMU-TBD/psi/tree/dev/k4abt-1.1

Here's what I changed:

1. The tracker configuration now lets you set whether to use the lite model and different type of runners (cpu, tensorRT, etc)
2. Updated the sensor nuGet to 1.4.1 and use path referencing for the body tracking SDK.
3. Used the environment variable AzureKinectBodyTrackingSDKDir to point to where you install the AzureKinectBodyTrackingSDK. This leads to the copying and hinting of dlls.
4. Added a try/catch block that solves a bug in the AzureKinectBodyTrackingSDK where an exception is always through at the beginning.

Unfortunately, this is a Windows only fix right now. The Body Tracking SDK team did release the package for Ubuntu 18.04 but I haven't try anything there yet.

Some of changes feel hacky to me and probably have better alternatives. Let me know if you think I should do a pull request on this. I'm also adding this to my own projects and we'll see if anything else breaks 😃

[xiangzhi]

Found the first downside of my approach. Any other project would need to add the following reference to use anything in the Microsoft.Azure.Kinect.BodyTracking namespace

  <ItemGroup>
    <Reference Include="Microsoft.Azure.Kinect.BodyTracking">
      <HintPath>$(AzureKinectBodyTrackingSDKDir)\sdk\netstandard2.0\release\Microsoft.Azure.Kinect.BodyTracking.dll</HintPath>
    </Reference>
  </ItemGroup>