Implement DRAKE_VIEWER_COMMAND: starting/stopping recording etc.

[wxmerkt]

• drake-visualizer currently does not subscribe to DRAKE_VIEWER_COMMAND
• as such, commands for recording movies e.g. triggered via visualizer.playbackMovie() (cf. here) are not being acted upon

I'd like to:

1. In drake-visualizer, subscribe to DRAKE_VIEWER_COMMAND to start and stop recording of movies programmatically
2. Extend the bot_core_viewer_command_t (here) enum for SAVE_SCREENSHOT- to my understanding this won't affect the message hash, so is backwards-compatible
3. Extend the command also for setting the view (camera position), with the serialised data being sent in the command_data field
4. Similarly, lock the view size/set the view size

The idea here is to be able to always set the visualizer to the same view, and remotely trigger screenshots and movie frame recordings. Alongside these changes will be changes to Drake's BotVisualizer to allow for these commands to be sent.

Very happy to hear input, comments, and feedback before I get to work - Thanks :)

[patmarion]

Sounds good to me, let me know if you need help on the director side. I think all of this functionality has existing python apis in director.

[wxmerkt]

Thanks! I'll ping you once I'm moving forward with this (not currently taking action on this but it's on the TODO)

[wxmerkt]

Hi @patmarion, I've picked this back up and started a quick prototype - cf. https://github.com/RobotLocomotion/director/compare/master...openhumanoids:wxm-extend-drake-visualizer-remote-addressing-280.

Regarding implementation, would you prefer having a new class DirectorLCMControl/DirectorLCMRemote such that it's neater and can be instantiated from the app class?

And would you like to discuss on conventions for the command_data? My thoughts are:

• type: SAVE_SCREENSHOT, data: absolute path + filename, if not given it defaults to ~/Pictures/Director/Screenshot-datetime.png
• type: SET_VIEW_SIZE, data: [width]x[height]
• type: SET_CAMERA_POSE, data: tbd - pose and direction
• type: START_RECORDING, data: absolute directory path - should it default to overwrite if it already exists given or fail?

Thanks, Wolf

[patmarion]

hey, have you seen https://github.com/RobotLocomotion/director/pull/420 and are aware that @rdeits has designed a new lcm drawing protocol? It will be merged soon. What is is currently missing is an interface library for clients in Python and C++, @rdeits has a client interface in Julia.

So while we could certainly add camera and screenshot commands to the drake draw protocol, it would be much easier to add it to the new protocol in https://github.com/RobotLocomotion/director/pull/420, but that would only help you if you switched to use that.

[wxmerkt]

Awesome, happy to switch and contribute to it if I can, e.g. with some work on the Python interface :) Do you think we can also add some of the proposed remote commands to the older visualizer?

[My use case is to save annotated input from an algorithm when something unpredictable happens for later analysis]

[patmarion]

Yes, I'd be happy to accept a PR that modifies the existing drakevisualizer python code to handle the new proposed protocol. I agree that adding new enum values to the control codes could be accepted, but you'll have to get the reviewed by the drake team.

I think your proposal sounds good. For SET_CAMERA_POSE, you might consider a "look at" style where to send the camera position and look target, and view up vector: https://www.opengl.org/sdk/docs/man2/xhtml/gluLookAt.xml and director supports these inputs via transformUtils.getLookAtTransform(lookAtPosition, lookFromPosition, viewUp)