FlowFuse
Product MQTT payload schemas recommendations for Team Broker
Description
We offer smart suggestions for payload structure for third-party brokers. This provides full UNS/MQTT schemas when a customer integrates in their third-party brokers to the FlowFuse UI (e.g. Hive MQ)
However, this is not supported for our in-built Team Broker, and this is leaving customers thinking that the payload schema generation work is broken.
Related Epic:
https://github.com/FlowFuse/flowfuse/issues/4788
Customers Impacted
- Raised by https://app-eu1.hubspot.com/contacts/26586079/record/0-2/10177258183
Which customers would this be available to
Team + Enterprise Tiers (EE)
Have you provided an initial effort estimate for this issue?
I have provided an initial effort estimate
I know there are some technical difficulties around this, but I'll let @knolleary expand
https://github.com/FlowFuse/flowfuse/issues/5241 is potentially an alternative, but would provide different UX across the Team/Third-Party Brokers, which isn't ideal.
When a client publishes to the team broker, the broker makes an ACL check on the forge app to verify the client is allowed to publish to the given topic. We take advantage of that call to maintain a list of active topics being used by the team without having to do any extra work or put any additional load on the broker.
When we added the 3rd party broker support, the only way to get the topic information was by subscribing to # and receiving everything. The broker agent could then record both topic and payload schema.
To get the payload schemas for the team broker, we'd no longer be able to take advantage of the acl check method; we'd need to spin up a broker agent for each team using the broker as we do for the 3rd party broker. In terms of scaling, this doubles the traffic our broker has to handle - as it has to publish every payload to the agent. If a user has no interest in the schema information, thats a load that is wasted.
I'd prefer a solution where a user opts into schema detection for a period of time, so we only need to capture the data stream if it's something the user wants to see.
