double-take [BUG] Custom snapshot only works if standing by camera for several seconds

Describe the bug I've enabled the custom image, which comes from the snapshot of my doorbell camera. I want the camera snapshot since the image quality is higher than the Frigate snapshot and thus less likely to give me false results in Doubletake.

I've noticed that the custom snapshot only works if standing by camera for several seconds. When I enable regular snapshots/latest from Frigate it seems to trigger matches almost instantly. Because of this, I'm losing out on most matches with the custom image snapshot. People come to my door but they don't show up in the Doubletake matches unless they stand at the door in approx the same spot for 7-10 seconds +.

Version of Double Take v1.12.1

Expected behavior Custom snapshots to work faster than they do today. I'd hope for it to be almost as fast or as fast as the Frigate snapshot/latest/mqtt.

Screenshots N/A

Hardware

OS: Docker/ Debian
Browser: N/A

Additional context The snapshot it's taking is from my Amcrest AD410 doorbell. The snapshot image size is 1280 × 960 and averages at 130KB.

config.yml

# Double Take
mqtt:
  host: 192.168.0.22
  username:
  password:
  client_id:
  topics:
    # mqtt topic for frigate message subscription
    frigate: frigate/events
    #  mqtt topic for home assistant discovery subscription
    homeassistant: homeassistant
    # mqtt topic where matches are published by name
    matches: double-take/matches
    # mqtt topic where matches are published by camera name
    cameras: double-take/cameras

frigate:
  url: http://192.168.0.22:5000
  # if double take should send matches back to frigate as a sub label
  # NOTE: requires frigate 0.11.0+
  update_sub_labels: true
  # stop the processing loop if a match is found
  # if set to false all image attempts will be processed before determining the best match
  stop_on_match: true
  # object labels that are allowed for facial recognition
  labels:
    - person
  attempts:
    # number of times double take will request a frigate latest.jpg for facial recognition
    latest: 0
    # number of times double take will request a frigate snapshot.jpg for facial recognition
    snapshot: 15
    # process frigate images from frigate/+/person/snapshot topics
    mqtt: false
    # add a delay expressed in seconds between each detection loop
    delay: 0
  image:
    # height of frigate image passed for facial recognition
    height: 640
  events:
    frontdoor:
      attempts:
        latest: 0
        snapshot: 15
        mqtt: false
        delay: 0
      image:
        height: 640
        snapshot: http://admin:[email protected]/cgi-bin/snapshot.cgi?

  
detectors: 
  compreface:
    url: http://192.168.0.22:8001
    key: XXXXXX-XXXXXX-XXXXXX-XXXXXX
#  deepstack:
#    url: http://192.168.0.22:5001
#    key: 
    
cameras:
  frontdoor:
    detect:
      match:
        save: true # save matches
        confidence: 80 # confidence to be a good match (depends on which provider you use)
        min_area: 3000 # min area of face box to have a good match
      unknown:
        min_area: 1500 # min area to keep an unknown match
        confidence: 79 # less than this confidence is considered "unknown"

Logs

2022-08-09T19:46:20.188104938Z   misses: []
2022-08-09T19:46:20.188113986Z }
2022-08-09T19:46:22.925113699Z info: done processing frontdoor: 1660074373.660844-26nvcm in 7.46 sec
2022-08-09T19:46:22.925354221Z info: {
2022-08-09T19:46:22.925363237Z   id: '1660074373.660844-26nvcm',
2022-08-09T19:46:22.925366131Z   duration: 7.46,
2022-08-09T19:46:22.925368238Z   timestamp: '2022-08-09T19:46:22.924Z',
2022-08-09T19:46:22.925370338Z   attempts: 2,
2022-08-09T19:46:22.925372321Z   camera: 'frontdoor',
2022-08-09T19:46:22.925374303Z   zones: [],
2022-08-09T19:46:22.925376248Z   matches: [
2022-08-09T19:46:22.925378190Z     {
2022-08-09T19:46:22.925380250Z       name: 'naomi',
2022-08-09T19:46:22.925382276Z       confidence: 93.4,
2022-08-09T19:46:22.925384272Z       match: true,
2022-08-09T19:46:22.925386239Z       box: [Object],
2022-08-09T19:46:22.925388216Z       type: 'snapshot',
2022-08-09T19:46:22.925390173Z       duration: 3.64,
2022-08-09T19:46:22.925392143Z       detector: 'compreface',
2022-08-09T19:46:22.925394094Z       filename: '22b7c4e8-27a8-457c-ad13-ddd3c144d3c2.jpg'
2022-08-09T19:46:22.925396132Z     }
2022-08-09T19:46:22.925398381Z   ],
2022-08-09T19:46:22.925400385Z   misses: []
2022-08-09T19:46:22.925402333Z }
2022-08-09T19:59:00.030366217Z info: purged 1 file(s)
2022-08-09T22:01:00.080407265Z info: purged 1 file(s)
2022-08-09T22:21:00.071900833Z info: purged 1 file(s)
2022-08-10T13:14:00.049507125Z info: purged 1 file(s)
2022-08-10T15:16:19.058890330Z info: processing frontdoor: 1660144577.565847-hpgbor
2022-08-10T15:16:53.959099570Z info: done processing frontdoor: 1660144577.565847-hpgbor in 34.9 sec
2022-08-10T15:16:53.961123473Z info: {
2022-08-10T15:16:53.961165645Z   id: '1660144577.565847-hpgbor',
2022-08-10T15:16:53.961179411Z   duration: 34.9,
2022-08-10T15:16:53.961189569Z   timestamp: '2022-08-10T15:16:53.957Z',
2022-08-10T15:16:53.961199292Z   attempts: 10,
2022-08-10T15:16:53.961208737Z   camera: 'frontdoor',
2022-08-10T15:16:53.961218221Z   zones: [],
2022-08-10T15:16:53.961227209Z   matches: [],
2022-08-10T15:16:53.961236089Z   misses: []
2022-08-10T15:16:53.961245084Z }
2022-08-10T15:52:16.265443096Z info: processing frontdoor: 1660146735.840877-9eqctt
2022-08-10T15:52:41.368683041Z info: done processing frontdoor: 1660146735.840877-9eqctt in 25.1 sec
2022-08-10T15:52:41.369533148Z info: {
2022-08-10T15:52:41.369729599Z   id: '1660146735.840877-9eqctt',
2022-08-10T15:52:41.369745858Z   duration: 25.1,
2022-08-10T15:52:41.369756668Z   timestamp: '2022-08-10T15:52:41.368Z',
2022-08-10T15:52:41.369766558Z   attempts: 10,
2022-08-10T15:52:41.369775837Z   camera: 'frontdoor',
2022-08-10T15:52:41.369784972Z   zones: [],
2022-08-10T15:52:41.369794139Z   matches: [],
2022-08-10T15:52:41.369803350Z   misses: []
2022-08-10T15:52:41.369812517Z }
2022-08-10T15:57:03.646147768Z info: processing frontdoor: 1660147022.58646-ey7eeb
2022-08-10T15:57:33.421058005Z info: done processing frontdoor: 1660147022.58646-ey7eeb in 29.77 sec
2022-08-10T15:57:33.421992939Z info: {
2022-08-10T15:57:33.422027672Z   id: '1660147022.58646-ey7eeb',
2022-08-10T15:57:33.422040516Z   duration: 29.77,
2022-08-10T15:57:33.422049993Z   timestamp: '2022-08-10T15:57:33.420Z',
2022-08-10T15:57:33.422060676Z   attempts: 10,
2022-08-10T15:57:33.422069910Z   camera: 'frontdoor',
2022-08-10T15:57:33.422079133Z   zones: [],
2022-08-10T15:57:33.422088807Z   matches: [],
2022-08-10T15:57:33.422097982Z   misses: []
2022-08-10T15:57:33.422106945Z }
2022-08-10T15:59:37.696588334Z info: processing frontdoor: 1660147176.261751-kzxw2e
2022-08-10T16:00:03.985353304Z info: done processing frontdoor: 1660147176.261751-kzxw2e in 26.29 sec
2022-08-10T16:00:03.986198990Z info: {
2022-08-10T16:00:03.986344673Z   id: '1660147176.261751-kzxw2e',
2022-08-10T16:00:03.986356396Z   duration: 26.29,
2022-08-10T16:00:03.986365758Z   timestamp: '2022-08-10T16:00:03.984Z',
2022-08-10T16:00:03.986415073Z   attempts: 10,
2022-08-10T16:00:03.986426556Z   camera: 'frontdoor',
2022-08-10T16:00:03.986435552Z   zones: [],
2022-08-10T16:00:03.986444468Z   matches: [],
2022-08-10T16:00:03.986453399Z   misses: []
2022-08-10T16:00:03.986462205Z }
2022-08-10T17:29:00.046397975Z info: purged 1 file(s)
2022-08-10T18:48:06.478981971Z info: processing frontdoor: 1660157285.91942-8tac19
2022-08-10T18:48:31.785171075Z info: done processing frontdoor: 1660157285.91942-8tac19 in 25.3 sec
2022-08-10T18:48:31.786266746Z info: {
2022-08-10T18:48:31.786304434Z   id: '1660157285.91942-8tac19',
2022-08-10T18:48:31.786316896Z   duration: 25.3,
2022-08-10T18:48:31.786326335Z   timestamp: '2022-08-10T18:48:31.784Z',
2022-08-10T18:48:31.786336272Z   attempts: 10,
2022-08-10T18:48:31.786345336Z   camera: 'frontdoor',
2022-08-10T18:48:31.786354217Z   zones: [],
2022-08-10T18:48:31.786362891Z   matches: [],
2022-08-10T18:48:31.786371630Z   misses: []
2022-08-10T18:48:31.786380539Z }

Aug 11 '22 13:08 conorlap

Odds are it is because the snapshot from the doorbell has to be parsed and decoded. A connection also needs to be made to the doorbell meanwhile the frigate snapshots are included in MQTT message. It Also takes more time for CompreFace to scan higher resolution images.

I have the exact same doorbell, in my case I just setup frigate to use the main stream from the doorbell for detect and set

    detect:
      width: 1280
      height: 960

this does use some more CPU to resize the image but it works great for DoubleTake and my system can easily handle it.

Aug 30 '22 15:08 NickM-27

Any solution here? It happened to me as well. My override snapshot is 1920x1080 and the side is about 430K. Frigate has no problem to spot the person, and double take seems to receive the picture successfully as the log suggests. However, no processing or results were posted in double take later.

Sep 14 '22 22:09 redwood996

@conorlap have you tried doing what @NickM-27 suggested? He's correct in that there is more latency when using a custom snapshot since there's an external API call to download the image to then pass it to the detector. And if this raw image is much larger, the overall time will increase.

I have Frigate config set to have a larger MQTT image for my cameras, since the default is only 270px.

mqtt:
      timestamp: False
      bounding_box: False
      crop: False
      height: 500

Oct 19 '22 04:10 jakowenko

Any solution here? It happened to me as well. My override snapshot is 1920x1080 and the side is about 430K. Frigate has no problem to spot the person, and double take seems to receive the picture successfully as the log suggests. However, no processing or results were posted in double take later.

Are you still having this issue? There isn't much logic in how this is done vs the normal Frigate snapshot image. Could it be a timing difference of the stream Frigate is reading vs how often your camera is updating the snapshot image?

Oct 19 '22 04:10 jakowenko

hey @jakowenko I've resorted back to mqtt image which seems to work quickly - the image is still pretty low res though since it's coming from my detect stream in Frigate. Thanks @NickM-27 for the info also, this issue can be closed 🙂

Oct 20 '22 12:10 conorlap

double-take double-take copied to clipboard

[BUG] Custom snapshot only works if standing by camera for several seconds

double-take
double-take copied to clipboard