Allow `Pump` and `Outlet` nodes to be controlled by allocation

[SouthEndMusic]

What After a refinement session, we (@SouthEndMusic, @gijsber, @visr) came to the conclusion that it would be nice to have a special control state "allocation", which is not defined via the static tables but means that the flow of a Pump or Outlet is set by allocation. However, when working out this issue so that @Jingru923 can collaborate on it, I came to some slightly different ideas on how to mark a node as being controlled by allocation, see below.

How

• [ ] Add a new optional string column to the static tables of Pump and Outlet named controlled_by, with allowed values "nothing" (default), "allocation" and "PID". We could set PID controllable nodes to PID by default to avoid a breaking change.
• [ ] Add a flag to Pump and Outlet nodes to indicate if and how they are controlled: The allocation algorithm must know which nodes are allocation controlled so that their flow_rate can be set. These nodes already have a boolean flag is_pid_controlled, so instead of adding a new flag, let's modify this one to an enumerator ControlType with values of the controlled_by options above.
• [ ] Compute and assign the flow_rate: after a run of the allocation algorithm, the flow through nodes that are allocation controlled must be computed and assigned. The assigned flow must be the sum over the priorities of the flow through the inflow edge of allocation controlled nodes. This sum can be computed during the allocation algorithm in pre-allocated memory, or be computed afterwards using the data in the flow_record (@visr let me know whether you think the latter is a good idea).
• [ ] Don't do PID control if not in that control type: Currently we assume that when PID control is available for a node, this node is always PID controlled (except when the PID control node is not active). If we want to incorporate PID control in the above (this could also be a follow-up issue), we need to check whether nodes are actually PID controlled when doing the PID part of water_balance!.

Validation

• Which conflicts between the different control types do we want to validate for?
• Allocation controlled nodes should at least be part of a subnetwork (and allocation should be active).
• If validation whether PID is available (see below).

Reasons why I prefer this over the original idea

• We don't have to implement and maintain special control states;
• Nodes can be assigned one of the control types at initialization without DiscreteControl getting involved;
• If desired,DiscreteControl can change the control type during simulation (for PID control it must be checked whether it is available for the node, preferably during initialization). Note that this means that DiscreteControl overarches the other control types, which is also why it is not one of the ControlType options.

[SouthEndMusic]

Also make sure that there are no conflicts between 'normal' control and control coming from allocation. This is probably a validation issue.

[gijsber]

Let's try whether allocation can update the setpoints for control instead of setting the control actions itself.

[visr]

For a subnetwork with an inlet to the main network, we want to:

1. sunetwork determine the desired inflow from the inlet, assuming max inlet capacity
2. top-level allocation subnetwork has all subnetworks aggregated as users, and sets the flow to the inlets
3. subnetwork allocates the flow over the inlet to the individual users

Flow over an inlet (Outlet or Pump) can be set by either allocation or control, but not both. Initially this can be up to the user to avoid conflicts, though perhaps we can add validation for this later.

[SouthEndMusic]

@visr this issue is not about connections between the main network and subnetworks, I propose moving that discussion to https://github.com/Deltares/Ribasim/issues/768.

[SouthEndMusic]

It came out of a discussion with @SnippenE @visr that a good way to approach this is to actually expand the capabilities of DiscreteControl, in the sense that conditions can be placed on allocation flows.

[visr]

For an downstream demand driven Outlet or Pump, we can support setting a special control state "allocation", which will set the flow_rate to the cumulative allocation flows over all priorities on the outcoming (or ingoing) edge.

We can add a validation rule that the control_state in DiscreteControl / logic is not allowed to be "allocation". If need be we could support that in the future.

[visr]

After discussing this together, I have the following proposal:

• We don't add a controlled_by column to static. We can track this internally as a ControlType enum. The network and our logic determines the ControlType. This helps to avoid specifying things in two places.
• If there is an active incoming PID controller, ControlType is PID.
• DiscreteControl can change ControlType by deactivating the PID controller.
• DiscreteControl can change ControlType to allocation by setting ribasim.allocation to be the control_state, irrespective of if that exists in the static table.
• If the static table has a control state of ribasim.allocation but no incoming control edge, ControlType is allocation.

[SouthEndMusic]

Here's a follow-up based on the discussion on how to proceed with allocation this year from yesterday with @gijsber and @SnippenE .

It seems from the discussion above that regardless of exactly how the allocation algorithm interacts with control in the physical layer, this happens always based on the cumulative flow over certain edges as computed by allocation. This information is available now, so this does requires neither level and storage states nor MPC. It's just that with these features the flows as computed by allocation are better informed.

Regarding the issue itself: it's been a while since we discussed the details of this, so I think it is good to determine what the minimal first concrete step towards this feature should be. Should allocation set pump/weir flow rates? That's the most readily available information. If allocation should set level setpoints, that requires some more design.

[SouthEndMusic]

As I see it now, there are 2 things we need to adress.

What allocation informed control looks like

Options:

• To me the most natural way to have control of pumps and outlets from allocation is to set the flow rate of these nodes as the flow rate required by allocation to achieve the allocations.
• Another option is to do it via discrete control, where allocation outputs are used as discrete control listen variables

How the various control types interact with eachother

The Pump and Outlet nodes are becoming rather complex. In the core, their flow rate can come from

• The flow_rate_itp (since https://github.com/Deltares/Ribasim/pull/2110)
• Either DiscreteControl, ContinuousControl or PidControl via the flow_rate_cache (renamed from flow_rate in the PR mentioned above)

The control via allocation is another addition to this. We have to either

• Decide and document a clear order of precedence for these different ways to set the flow rate of a single node if multiple are available, although this can be rather limiting
• Make a system where a flag for each pump and outlet states whether and how it is controlled (the control type enum). Then we have to figure out what the initial control type of nodes is. Discrete control should then be able to change the control type of a node.

[JohnRushKucharski-Deltares]

This comes after #2255