Add tests for IS-07

[garethsb]

The existing test suite for IS-07 (IS0701Test.py) is just a placeholder, with the 'auto' tests, i.e. those few that can be generated automatically from the RAML of the Events API.

Here's a start on identifying the additional tests that would be useful:

IS-07 Events API

• [ ] JSON Schema validation (if we haven't got that for free?)
• [ ] Events API /state endpoint is consistent with the /type (type definition object) endpoint, e.g.
  • [x] ids correspond
  • [x] event types (state is valid sub-type of event type in the type definition object)
  • [x] numeric values comply with min/max/step criteria (mandatory or recommended?)
  • [x] numeric values have consistent scale values (recommended)
  • [ ] numeric event types and measurement units correspond
  • [x] string values comply with any specified min/max length criteria
  • [x] string values comply with any specified regex pattern
  • [x] enum values are found in the type definition object
• [x] In the Events API, "flow_id" must not be included in the "identity"

The bulk of the tests need to to consider consistency/interaction between IS-04, IS-05 and IS-07, however. This is described in the IS-07 Core models. This is slightly related to #152, but I'm thinking more in the style of IS0502Test.py.

IS-04 Node API

• [x] IS-04 Sources/Senders/Flows attributes match attributes in IS-07 Events API Sources
• [ ] IS-04 Sources/Senders/Flows attributes match attributes in IS-07 Events WebSocket API messages
• [ ] Sender manifest_href is null (see https://github.com/AMWA-TV/nmos-event-tally/issues/38)

IS-05 Connection API

Senders

• [ ] IS-05 Senders match IS-07 Sources and are self-consistent (e.g. between broker_topic or connection_uri and the ext_ parameters, and between /active, /staged and /constraints)
• [ ] IS-07 Events WebSocket API server does not accept connections if master_enable is false
• [ ] In the case of multiple Sources/Senders, they should share the same WebSocket server URL at least within each Device
• [ ] /transporttype is as expected for IS-07
• [ ] /transportfile endpoint is 404 (see https://github.com/AMWA-TV/nmos-event-tally/issues/36, https://github.com/AMWA-TV/nmos-device-connection-management/pull/72)
• [ ] PATCHing /staged with parameter values that don't comply with the constraints is rejected
• [ ] PATCHing connection_uri with "auto" is accepted and takes the value from constraints

Receivers

We're going to need mock some pieces of an IS-05 Sender / IS-07 Source / WebSocket server in the test suite...

IS-07 Events WebSocket transport

• [x] WebSocket connection is automatically closed by the server if no command is sent for 12 seconds after it is opened
• [x] Heartbeat health commands keep the WebSocket connection open
• [x] Heartbeat health commands result in a health response message
• [x] No state messages are received until after a subscription command
• [x] A state messages is 'immediately' received after a subscription command (one entry in sources)
• [ ] Repeating a subscription command, even with the same id, causes the state message to be resent
• [ ] State messages stop being received 'soon' after a subscription command with an empty sources array
• [ ] JSON Schema validation of messages
• [ ] Each state message has the correct identity information including both source_id and flow_id
• [ ] State messages are consistent with the Events API /type (type definition object) endpoint
• [ ] If there are multiple sources, a subscription command with more than one entry in sources causes a state message to be sent for each source
• [ ] If multiple sources are subscribed to, each state message has the correct identity information including both source_id and flow_id
• [ ] If multiple sources are subscribed to, heartbeat health commands still result in only one response
• [ ] Invalid WebSocket message, e.g. binary (see https://github.com/AMWA-TV/nmos-event-tally/issues/41)
• [ ] Badly formatted commands (also https://github.com/AMWA-TV/nmos-event-tally/issues/41)
• [ ] Behaviour when a random source id is included in subscription command (also https://github.com/AMWA-TV/nmos-event-tally/issues/41)

IS-07 Events MQTT transport

We're presumably going to need a mock MQTT broker and client in the test suite?

[cristian-recoseanu]

Here is a link to at least one python MQTT broker library

https://hbmqtt.readthedocs.io/en/latest/references/broker.html