Access camera params (extrinsics, intrinsics, near/far plane) for pointcloud calculation

[stepjam]

Hello, Is there a way to get either access to a point-cloud observation or access to the camera params (extrinsics, intrinsics, near/far plane) such that I can calculate the point-cloud from z-buffer observation? Thanks!

[charlesbeattie]

You can get information about the camera position with game:playerInfo. I believe the default vertical field of view is 90. You can calculate the horizontal FOV using the aspect ratio. You can use ray casts from the players eye position to generate a point cloud. See game:raycast.

Player look direction and posion:

local game = require 'dmlab.system.game'
local info = game:playerInfo()
local look = info.angles
local eye = info.eyePos

Raycast see: https://github.com/deepmind/lab/blob/2f21c07371e0f2a2c8e6a039b330c44b97d418ff/game_scripts/levels/tests/raycast_test.lua#L74