Synchronizing Coordinate Systems in Holographic Remoting

[xyq3q]

Hi,

Your work is truly commendable, and I appreciate your efforts.

I have integrated hl2ss into the Holographic Remoting Sample Player and am using Unity for remote rendering. I am curious about how to synchronize Unity's coordinate system with that of hl2ss. I am aware of using UpdateCoordinateSystem() in UWP applications, but I'm unsure how to implement this in Holographic Remoting. I look forward to your guidance on this matter.

Best regards

[jdibenes]

Hi,

I cannot test this right now so I'm not sure it will work but:

1. Uncomment #define ENABLE_USER_COORDINATE_SYSTEM_SAMPLE in SamplePlayerMain.h
2. In SamplePlayerMain.cpp, in SamplePlayerMain::Initialize (lines 433-434) change m_userSpatialFrameOfReference = m_spatialLocator.CreateStationaryFrameOfReferenceAtCurrentLocation(float3(0.5f, 0.0f, -2.0f), quaternion(0, 0, 0, 1), 0.0); to m_userSpatialFrameOfReference = m_spatialLocator.CreateStationaryFrameOfReferenceAtCurrentLocation();
3. In SamplePlayerMain.cpp in SamplePlayerMain::SetWindow after InitializeStreams (hl2ss) add OverrideWorldCoordinateSystem(winrt::get_abi(m_userSpatialFrameOfReference.CoordinateSystem()))

This is based on https://learn.microsoft.com/en-us/windows/mixed-reality/develop/native/holographic-remoting-coordinate-system-synchronization-wmr I'm hoping Unity internally calls m_remoteContext.GetUserSpatialFrameOfReference(); (or equivalent) when remoting, but I don't know if that's the case.

[andreclerigo]

Hi @xyq3q, I am actually very curious on how you managed to integrate hl2ss with MRTK's Holographic Remote Player. Does that mean you can run an application through the Holographic Remote Player and still have access to things like the Video Camera Feed? I would be really glad if you explained how you achieved that.

[xyq3q]

Hi @xyq3q, I am actually very curious on how you managed to integrate hl2ss with MRTK's Holographic Remote Player. Does that mean you can run an application through the Holographic Remote Player and still have access to things like the Video Camera Feed? I would be really glad if you explained how you achieved that.

Yes, you're correct that it's possible to run an application through the Holographic Remote Player while still accessing features like the Video Camera Feed. Initially, I integrated these two components myself, but later I found that the author jdibenes has also done the integration. You can check out this repository for more details: https://github.com/jdibenes/MixedReality-HolographicRemoting-Samples-hl2ss.