mavlink icon indicating copy to clipboard operation
mavlink copied to clipboard

GCS Operator control updates

Open hamishwillee opened this issue 5 months ago • 9 comments

This updates the MAV_CMD_REQUEST_OPERATOR_CONTROL command and CONTROL_STATUS message to support simultaneous control of a system by multiple GCS. This still allows use cases where pilots transfer control, but now they can also share simultaneous control.

As discussed in https://github.com/ArduPilot/ardupilot/pull/30627 this allows cases where you have multiple pilots co-located so that they can safely work together at the same time. The benefit of the model is that if one controller goes down, another can takeover without the vehicle losing its GCS (as long as any GCS that is an owner is still connected).

@Davidsastresas Let's iterate a bit to our own satisfaction. I'm getting tired, so this needs to be checked. Fyi @auturgy @julianoes

hamishwillee avatar Jul 17 '25 07:07 hamishwillee

I don't think the sysid range concept should be locked into the mavlink standard, instead it should be an implementation detail where there is a variable for which of the IDs in the range is currently active, default would be all active. So we can still do the takeover API, but it would be limited in the ArduPilot implementation to IDs in the range

tridge avatar Jul 28 '25 23:07 tridge

Thank you very much @hamishwillee. I commented over each thread, but overall it looks good to me.

@tridge Can you elaborate more? I don't think I understand correctly what you mean.

We also have this comment from @peterbarker in https://github.com/ArduPilot/ardupilot/pull/29252#issuecomment-3135273941

Hit an interesting thing where we want multiple GCSs to be considered "mygcs" - but do not want more than one to be able to provide RC inputs to the vehicle.

Which has a point. If all GCS in the range are sending manual_control or RC_override it will collide.

In a setup where all the operators are working closely, it is probably fine as it is straighforward to just use a Joystick or manual controls just in one of the GCS, so the rest of them won't even send those manual_control or rc_override messages.

To be honest I think including this in the protocol would be too involved, and we would end up having a whole control request thing between our range of controlling GCS, which seems really convoluted to me.

This new changes for a range of GCS instead of single GCS in control was based on the idea of very good communication on the operators of the same "group of GCSs". My opinion would be to rely on operator understanding and setup for this front of manual control.

I would like to hear your thoughts on it though. Is this something we should warn over in the docs? Do we want to make something about it?

Davidsastresas avatar Jul 30 '25 11:07 Davidsastresas

@tridge I don't get it either. The idea here is that the request sets the ranges mapped in parameters or whatever in the flight stack. You may be suggesting that the flight stack co-exists with those parameters - i.e. the ranges are set hard and we advertise which can be used.

One model I considered, which may be what you are thinking, is that instead of requesting a range each GCS requests for itself for exclusive or shared ownership.

  • If the exclusive is set then ownership is granted and the flight stack sets the upper and lower bounds to match.
  • If the shared ownership specified then it is automatically granted to a GCS in the range allowed by the flight stack parameter.

Could be completely wrong.

@Davidsastresas agree with you on the RC/manual commands. The justification for having the range was that one computer goes down then commands can still be accepted from the other. That's still true for RC/manual_control. In any case, I don't see how it can work with the current model unless the bits controlled by the manual_control are independent mavlink components (that is supported by the current model)

hamishwillee avatar Jul 31 '25 04:07 hamishwillee

On Wed, 30 Jul 2025, Hamish Willee wrote:

@Davidsastresas agree with you on the RC/manual commands. The justification for having the range was that one computer goes down then commands can still be accepted from the other. That's still true for RC/manual_control. In any case, I don't see how it can work with the current model unless the bits controlled by the manual_control are independent mavlink components (that is supported by the current model)

The sad bit is that you probably want normal stuff (arm / mission upload / set-waypoint) stuff to be allowable from any GCS. But you probably do NOT want more than one GCS to provide "streaming" commands like RC_CHANNELS_OVERRIDE or MANUAL_CONTROL or SET_ATTITUDE_TARGET.

The scenario tridge hand-waved together at DevCall was a pair of MissionPlanner GCSs with joysticks attached. There's no way the MissionPlanners will mutex the use of those joysticks, so it's down to the autopilot to avoid the problem.

One resolution might be "lock onto whoever send you stuff first and they have exclusive control for that functionality until they stop sending stuff and it times out". I thought of but didn't raise it in the meeting as I haven't thought that one through.... I think the lack of a positive "you have control"/"I have control" thing is the biggest drawback, leading to, "why can't I steer this vehicle?!" followed at length by, "then how do I stop controlling it with my joystick?" followed by, "I wonder where our vehicle is".

peterbarker avatar Aug 02 '25 00:08 peterbarker

On Wed, 30 Jul 2025, Hamish Willee wrote: @Davidsastresas agree with you on the RC/manual commands. The justification for having the range was that one computer goes down then commands can still be accepted from the other. That's still true for RC/manual_control. In any case, I don't see how it can work with the current model unless the bits controlled by the manual_control are independent mavlink components (that is supported by the current model) The sad bit is that you probably want normal stuff (arm / mission upload / set-waypoint) stuff to be allowable from any GCS. But you probably do NOT want more than one GCS to provide "streaming" commands like RC_CHANNELS_OVERRIDE or MANUAL_CONTROL or SET_ATTITUDE_TARGET. The scenario tridge hand-waved together at DevCall was a pair of MissionPlanner GCSs with joysticks attached. There's no way the MissionPlanners will mutex the use of those joysticks, so it's down to the autopilot to avoid the problem. One resolution might be "lock onto whoever send you stuff first and they have exclusive control for that functionality until they stop sending stuff and it times out". I thought of but didn't raise it in the meeting as I haven't thought that one through.... I think the lack of a positive "you have control"/"I have control" thing is the biggest drawback, leading to, "why can't I steer this vehicle?!" followed at length by, "then how do I stop controlling it with my joystick?" followed by, "I wonder where our vehicle is".

@joshanne actually reports they do have the scenario tridge mentioned.

peterbarker avatar Aug 04 '25 00:08 peterbarker

One model I considered, which may be what you are thinking, is that instead of requesting a range each GCS requests for itself for exclusive or shared ownership.

Problem is that you probably want a mixed ownership model here. Where some GCSs can execute commands (and have their heartbeats considered to be acceptable to avoid triggering a GCS failsafe), but only one GCS can have its MANUAL_CONTROL messages consumed by the vehicle.

This makes the "you have control / I have control" phraseology problematic. However, in a manned cockpit that's referring to manual pilot input - the pilot-not-flying is still allowed to press buttons!

peterbarker avatar Aug 04 '25 00:08 peterbarker

All this time back Hamish and I didn't thought of this "co-ownership" concept. It is unfortunate because we iterated a lot and I think we left it pretty polished for the initial goals, and nobody really gave feedback back then.

I think we are all exposing good points, but I feel it is getting too complicated very quickly. Maybe we can take a couple of steps back and discuss an alternative approach:

  • In the current approach autopilot can have a range of system ID as valid GCS. What if instead we rely on both GCS having the same system ID, but different compid? Technically is not too wrong I think, it is a group of GCS and operators really close together, and one might be pilot, other mission planner, other camera operator, etc. I think it could make sense.

  • From that group of GCS, with same systemid but different compid, one of them is the "leading GCS", and it is the only one able to:

  1. Send manual control commands ( rc override, manual control, etc )
  2. Receive control requests from other GCS with different system id. This "leading GCS" is the one that will receive such notifications.
  • Then from the autopilot point of view we could manage giving control to other GCS compid if the current "leading GCS" goes out of scope. We could give this "leading GCS role" to the first GCS compid that sends such manual control commands, or something else, we could change this behavior playing around with parameters.

I think something like that could maybe save some complexity? What do you think of exploring that path instead?

Thanks.

Davidsastresas avatar Aug 04 '25 16:08 Davidsastresas

We've iterated through one thing in control on one thing, to multiple things with control over one thing, to multiple things potentially in control over different things, some exclusively and some shared.

I don't "believe" this use case - I've seen flying high - pilot and co pilot have a joystick. They seem perfectly capable of not trying to control each other's stick - or press each other's buttons :-)

@Davidsastresas I'm not sure how this idea would simplify things.

One way to think about this is to imagine how you would implement it in the absence of MAVLink. What I'd do is have parameters for the GCS that are allowed to be in control and send commands generally, and then another (or others) for things that must be controlled by only one GCS - in this case movement commands.

Assuming we only need one thing that could be separately controlled you could use param 7 for the GCS in control of movement, and add a field for this in the CONTROL_STATUS. Everything else would be the same except on losing the GCS in charge of movement you'd probably allow first other GCS to send one of the messages to latch, if it is in the set of allowed GCS.

Assuming we might need more than one thing that could be separately controlled then maybe a bit like discussed in https://github.com/mavlink/mavlink/pull/2313#issuecomment-3138531543, In this case you'd have every GCS that wants control to request it separately, specifying its own system ID, and you'd change the Takeover allowed boolean to a set of flags indicating whether takeover is allowed, what you want to take over, whether it is shared or exclusive access. You'd probably still set the allowed GCS range in parameters so you might need to advertise that in the CONTROL_STATUS, along GCS array for anything that gets exclusive access. Details would have to be worked out.

hamishwillee avatar Aug 06 '25 03:08 hamishwillee

@Davidsastresas Here is my proposal to meet both use cases. If you think it makes sense we can push this forward. The benefit is that nothing changes for your mode! There is another mode for shared access which behaves similarly. As an implementer you could choose not to implement support for that mode IMO without breaking anything. But this does give a path forward for those who want the shared access.

Requirements

For the exclusive ownership of all commands.

  • Advertise what GCS is in charge
  • We want to mediate handover via allowing takeover
  • Any GCS should be allowed to take over if there is no ownership - i.e. in this case a request would set the owner parameter low/hi under the hood

Shared ownership with exclusive ownership of movement:

  • Advertise what GCS values are allowed to send "non exclusive control"
    • Any GCS with an ID in that range can send any command such as arm etc., but not necessarily the stick movement commands.
    • These ranges are set in the parameters of the flight stack.
    • Unlike for the exclusive ownership you can't request ownership - you just have it if your id is correct
    • Probably can't advertise 255 items - message not big enough if we shove these in an array
  • Only one GCS can control movement.
    • Under-the-hood, presumably this is a parameter.

Proposal

I think we can do this as two distinct modes in the flight stack: exclusive/shared. This could be configured by a parameter - you might even use existence of a range such as SYSID_HI to configure this.

That allows this to be implemented almost as currently designed for exclusive mode, and people can choose if they want to implement both modes. In other words, it can be implemented without requiring that it works for both cases initially.

    <message id="512" name="CONTROL_STATUS">
      <description>Information about GCS(s) in control of this MAV.
        This should be broadcast at low rate (nominally 1 Hz) and emitted when ownership or takeover status change.
		The MAV has two modes of control: exclusive-control and shared-control.
		
		Exclusive-control mode is a model where only one GCS can be in control. 
	    It is designed to support safe handover of control, either automatically or requiring permission.
		In exclusive control mode all components should accept any and all commands only from the GCS with the ID listed in sysid_exclusive_control.
		The flags field indicates whether another GCS that requests takeover should automatically be granted ownership, or whether this should be requested of the current owner.
        Control over the MAV is requested using MAV_CMD_REQUEST_OPERATOR_CONTROL.
		The sysids_shared_control should be all-zero in this case. 
		
		Shared-control mode is designed to support the case where a number of GCS are in control, so control is not lost if one breaks.
		At the same time, only one GCS should be sending movement commands because there is no easy way for the autopilot to multiplex these.
		In shared control mode the sysids_shared_control array lists all GCS that are allowed to control the GCS for commands other than movement AND that are currently connected/alive as indicated by the HEARTBEAT.
		The GCS in this set are defined internally by the flight stack: they can't be requested or handed over. 
		The sysid_exclusive_control id indicates the GCS that is allowed to send any command, including movement commands. 
		The id must also be present in the sysids_shared_control.
		Ownership is managed in the same way as exclusive control in exclusive mode, with the caveat that ownership can only be handed to a GCS listed in sysids_shared_control.
		In addition, if ownership is lost, it may also be automatically gained by the first autopilot in sysids_shared_control that moves its control sticks.
		Note that sysids_shared_control is limited to 10 items: if more are enabled by the autopilot they will not be displayed.
</description>
      <field type="uint8_t" name="sysid_exclusive_control">System ID of GCS in (exclusive) manual control. If sysids_shared_control has values this is the GCS with "manual control". If sysids_shared_control is empty this is the exclusive controller. 0: no GCS in manual (or exclusive) control.</field>
      <field type="uint8_t" name="flags" enum="GCS_CONTROL_STATUS_FLAGS">Control status. For example, whether takeover is allowed, and whether this message instance defines the default controlling GCS for the whole system.</field>
      <field type="uint8_t[10]" name="sysids_shared_control">System IDs of GCS(s) that are allowed to send "shared-control_commands" In shared control must include the . Empty if in "exclusive_control". only Must include value in sysid_exclusive_control. In Set empty if exclusive control enabled. Must include sysid_manual_control value if allowing shared control. This should only display active GCS that are allowed to request control.</field> 
    </message>
    <enum name="GCS_CONTROL_STATUS_FLAGS" bitmask="true">
      <description>CONTROL_STATUS flags.</description>
      <entry value="1" name="GCS_CONTROL_STATUS_FLAGS_SYSTEM_MANAGER">
        <description>If set, this CONTROL_STATUS publishes the controlling GCS for the whole system. 
          If unset, the CONTROL_STATUS indicates the controlling GCS for just the component emitting the message.
          Note that to request control of the system a GCS should send MAV_CMD_REQUEST_OPERATOR_CONTROL to the component emitting CONTROL_STATUS with this flag set.
        </description>
      </entry>
      <entry value="2" name="GCS_CONTROL_STATUS_FLAGS_TAKEOVER_ALLOWED">
        <description>Takeover allowed (requests for control will be granted). If not set requests for control will be rejected, but the controlling GCS will be notified (and may release control or allow takeover).</description>
      </entry>
    </enum>

I haven't updated the docs for MAV_CMD_REQUEST_OPERATOR_CONTROL but they would be largely unchanged. In exclusive mode the command sets the owner, and this might change some matched parameter in the autopilot. However in the shared mode the parameters indicating the allowed GCS are fixed by the autopilot. So in this case the command simply selects one of the allowed GCS, if any.

hamishwillee avatar Aug 13 '25 07:08 hamishwillee