cockpit icon indicating copy to clipboard operation
cockpit copied to clipboard

Add viewer for 3D models and scenes

Open ES-Alexander opened this issue 2 years ago • 1 comments

Useful for things like

  • showing the current vehicle attitude
    • this could use the vehicle's active model when using BlueOS, and could likely make use of @Williangalvani's model viewing code from the BlueOS Vehicle Setup page
    • SpaceX have recently added this kind of functionality to their streams: Image
  • showing the vehicle/mission within its 3D environment
    • this is particularly useful if there's no camera feed, but there is a positioning system and a map with buildings and/or terrain data
    • this was recently demoed in QGC, using OpenStreetMap data
    • should support custom 3D area models as the source
    • should support projection mapping from an overhead / satellite image
      • even better if it also supports overlay / live projection/photogrammetry from a camera feed on the vehicle
    • could (ideally) support live updates from a source of surface distance data compared to the vehicle's barometric altitude/depth
      • for water-based use-cases this would need to take tide level (and waves?) into account

If done using .glb files this could likely integrate quite nicely with AR/VR viewers/headsets.

ES-Alexander avatar Aug 21 '23 08:08 ES-Alexander

Here's some ideas:

  • A 3D UI capable of showing the UAV live
  • Pose updates smoothly
  • Vehicle type sets the vehicel in the simulation (plane, copter, sub)
  • Shows all geofence data
  • Has a HUD (and tied to the map)
  • Views for 3rd person birds eye (follow), FPV front view, and the flight simulator 3rd person trailing, and completely free view
  • Easy to extend for custom non-mavlink messages (say a custom ROS path planner wants to visualize a special flight plan
  • Works well with ArduPilot, but can be used with other non-mavlink autopilots
  • For this, I am wondering if it makes sense to expose a non-mavlink message API
  • Ability to set GoTo waypoints
  • Ability to add custom user controls (IE, plan path from A to B using a ROS path planner)

Our companion computer has no way to visualize any data unless we use RVIZ. I want a tool that I can extend to visualize proprietary 3D data that has a solid foundational base capability that it's useful on its own.

Laastly, I need ability to visualize custom data from custom algorithms from the companion computer. This would likely require direct access to the cesium viewer object. Because the data may be proprietary, I would like to be sure the licensing of the code does not mean plugin developers modifying cockpit will have to release those modifications to their customers.

Ryanf55 avatar Nov 18 '24 20:11 Ryanf55