Implementation of runtime:packet and runtime:packetsent, bridge component (remote subgraph/runtime-to-runtime)

[ERnsTL]

Info:

• the event types of the runtime:packet message are the same as the network:{connect, begingroup, data, endgroup, disconnect} output messages
  • runtime:packet message is for remote subgraphs = runtime-to-runtime connection using FBP Network Protocol
  • network:... messages are for debugging to be sent to clients who request to listen to what is flowing on certain edges by subscribing to them using network:edges

What needs to be done:

• DONE graph exported ports implementation:
  • DONE delivery of packets to/from these graph ports
  • DONE ability to connect graph ports to/from components
• DONE delivery:
  • DONE delivery of the IP contained in an runtime:packet input message into the FBP network
  • DONE delivery of graph output IPs to FBP clients
• DONE Notifications of FBP clients:
  • DONE inports start: in runtime.start() add notification of clients that the graph inports are now connected
  • DONE outports start: in the graph outport handler thread when it starts, send notification to clients that the graph outports are now connected.
  • DONE inports stop: In runtime.stop() send notification, early, that the graph inports are now disconnected. This will happen before the outports handler is notified.
  • DONE outports stop: When outport handler thread exits its mainloop, send notification that the graph outports are now disconnected.

---

• TODO support for all event types: https://github.com/flowbased/fbp-protocol/blob/555880e1f42680bf45e104b8c25b97deff01f77e/schema/yaml/runtime.yml#L53

  • connect - DONE send when graph inports and outports are connected = on network start(), see above
  • disconnect - DONE send when graph inports and outports are disconnected = on network stop(), see above
  • TODO begingroup (opening bracket) - blocked by #256
  • TODO endgroup (closing bracket) - blocked by #256
  • DONE data

• TODO infrastructure to make a connection to a remote runtime as subgraph on a remote machine (need some configuration - where is the remote IP address stored for example?)

  • TODO relationship between subgraph <-> remote runtime IP+Port+Token
  • TODO directed/filtered traffic based on above relationship so that not all traffic flows to all connected FBP Network Protocol clients!
  • should this the runtime-to-runtime link be done and managed as a component (bridge component) or on a runtime level? This runtime is a client and we create a link to another runtime, which for example offers backend processing services. So we are the active part.
  • There is no trace in the protocol to manage this as at a built-in runtime level.
  • Easy to do as a bridge component.
  • TODO Need to map a variable number of input ports with graph inports of the remote runtime, best done using addressable port, for example port OUT -> IN[REMOTEIN] -> Bridge OUT[REMOTEOUT] -> IN ...
  • TODO Need to map a variable number of output ports with graph outports of the remote runtime.
  • TODO blocked on addressable/indexable ports #216 - but noflo-ui does not seem to have a way in the UI to connect an edge to a specific index (addressable ports behave the same as non-addressable ports!)
  • otherwise there is only a super-boring solution of IN1, IN2, IN3 and OUT1, OUT2, OUT3 and define mapping using CONF like IN1=REMOTEIN,OUT1=REMOTEOUT

Also needed (separate issue):

• How to handle reception of packets from unexpected ports = what if there are more graph outports on the remote runtime and we happen to receive an IP from one of these?
• TODO management of token <-> graph inports and graph outports subscription, meaning "if a client connects with that token, then he gets traffic only from these listed graph outports... and can only send to the allowed graph inports".
  • How to handle disconnection? either of the whole connection or disconnection of a graph outport on the remote runtime?
• TODO keep track of connection status and notify internally if disconnected etc.