frigate icon indicating copy to clipboard operation
frigate copied to clipboard

[Support]: Add event ID to audio events mqtt payload (Or use a simplified JSON payload based on camera events JSON)

Open Cheerpipe opened this issue 2 years ago • 0 comments

Describe the problem you are having

Frigate uses frigate/<camera_name>/audio/<audio_type> topic to notify audio events but it only publish ON/OFF data as payload making it impossible to create rich Home Assistant Notifications with a clip or with a link to the event to watch on frigate UI.

Frigate UI does show a snapshot and a clip in the Events UI so i assume the metadata, snapshot and clip exists. I know a snapshot may not be usefull for an audio event but the clip does, also the ID to create a link for the clip so the notifications can show the video and allow the user to navigate to the frigate UI.

I think the only missing value to acomplish what i want to do is the ID because with this i can recreate the snapshot and clip URL to use in mobile notifications.

Payload may be a simplified version of full event payload

{
  "type": "update", // new, update, end
  "before": {
    "id": "1607123955.475377-mxklsc",
    "camera": "front_door",
    "frame_time": 1607123961.837752,
    "snapshot_time": 1607123961.837752,
    "label": "person",
    "start_time": 1607123955.475377,
    "end_time": null,
    "thumbnail": null,
    "has_snapshot": false,
    "has_clip": false
  },
  "after": {
    "id": "1607123955.475377-mxklsc",
    "camera": "front_door",
    "frame_time": 1607123962.082975,
    "snapshot_time": 1607123961.837752,
    "label": "person",
    "start_time": 1607123955.475377,
    "end_time": null,
    "thumbnail": null,
    "has_snapshot": false,
    "has_clip": false
  }
}

I think all the above should/may exists for an audio event. I obviously removed the video related data.

I also think all audio events should/may be sent in a single topic instead of using camera name and label as part of the topic to allow users to create a single automation and use conditions to filter camera/label events.

Posible use cases

  • Create a notificatión with ongoing audio event (For ON/Before events)
  • Create or update existing notifications for audio events with video preview and video link (For OFF/After events).
  • Filter events by duration (with start/end data for OFF/After events or maybe a new duration in seconds field).
  • Extract audio from video and send to some endpoint (With de ID/URL of the clip).
  • Filter by event loudness (with the db that triggered the event. May be cool to have min/max/avg db of the event too).

Cheerpipe avatar Dec 05 '23 12:12 Cheerpipe