[Support]: How to reduce false positives?

[mcmara]

Describe the problem you are having

This issue is related to the reliability of AI object detection with Coral/OpenCV/Tensorflow. Almost all of the detected entities are mislabeled, or labaled with pure random objects. This is a typical list of events in my setup:

Is there a suggestion for making the detection more useful? I'm afraid that, in this way, it is no better than a random guess.

Version

0.11.1-2EADA21

Frigate config file

mqtt:
  host: homeassistant.local.<deleted>.<deleted>
  user: frigate
  password: <deleted>

# Optional: ffmpeg configuration
ffmpeg:
  # Optional: global hwaccel args (default: shown below)
  # NOTE: See hardware acceleration docs for your specific device
  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p

detect:
  width: 1280
  height: 720
  fps: 5

objects:
  # Optional: list of objects to track from labelmap.txt (default: shown below)
  track:
    - person
    - bicycle
    - car
    - motorcycle
    - bird
    - cat
    - dog
    - horse
    - sheep
    - cow
    - elephant
    - bear

birdseye:
  enabled: True
  mode: continuous

cameras:
  camera_1:
    ffmpeg:
      inputs:
        - path: rtsp://<deleted>:<deleted>@192.168.115.181:554/stream0
          roles:
            - record
            - detect
        - path: rtsp://<deleted>:<deleted>@192.168.115.181:554/stream1
          roles:
            - rtmp

  camera_2:
    ffmpeg:
      inputs:
        - path: rtsp://<deleted>:<deleted>@192.168.115.202:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
        - path: rtsp://xxx:[email protected]:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - rtmp
        - path: rtsp://<deleted>:<deleted>@192.168.115.202:554/cam/realmonitor?channel=1&subtype=2
          roles:
            - detect

detectors:
  coral:
    type: edgetpu
    device: usb

record:
  enabled: True
  events:
    retain:
      default: 10

Relevant log output

Nothing relevant here. All seems OK except the usual camera temporary hangs:

[ . . . ]
frigate    | [2022-12-08 00:00:40] watchdog.camera_2              INFO    : Terminating the existing ffmpeg process...
frigate    | [2022-12-08 00:00:40] watchdog.camera_2              INFO    : Waiting for ffmpeg to exit gracefully...
frigate    | [2022-12-08 00:01:00] watchdog.camera_2              INFO    : No frames received from camera_2 in 20 seconds. Exiting ffmpeg...
frigate    | [2022-12-08 00:01:00] watchdog.camera_2              INFO    : Waiting for ffmpeg to exit gracefully...
frigate    | [2022-12-08 00:01:00] frigate.video                  ERROR   : camera_2: Unable to read frames from ffmpeg process.
frigate    | [2022-12-08 00:01:00] frigate.video                  ERROR   : camera_2: ffmpeg process is not running. exiting capture thread...
frigate    | [2022-12-08 00:01:10] watchdog.camera_2              ERROR   : Ffmpeg process crashed unexpectedly for camera_2.
frigate    | [2022-12-08 00:01:10] watchdog.camera_2              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate    | [2022-12-08 00:01:10] ffmpeg.camera_2.detect         ERROR   : [AVHWDeviceContext @ 0x55cd0e6e1b40] libva: /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so init failed
frigate    | [2022-12-08 00:01:10] ffmpeg.camera_2.detect         ERROR   : rtsp://<deleted>:<deleted>@192.168.115.202:554/cam/realmonitor?channel=1&subtype=2: Connection timed out
frigate    | [2022-12-08 18:41:14] frigate.watchdog               INFO    : Detection appears to be stuck. Restarting detection process...
frigate    | [2022-12-08 18:41:14] root                           INFO    : Waiting for detection process to exit gracefully...
frigate    | [2022-12-08 18:41:44] root                           INFO    : Detection process didnt exit. Force killing...
frigate    | [2022-12-08 18:41:44] detector.coral                 INFO    : Starting detection process: 376772
frigate    | [2022-12-08 18:41:44] frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate    | [2022-12-08 18:41:46] frigate.edgetpu                INFO    : TPU found
frigate    | [2022-12-08 21:01:34] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:40168]
frigate    | [2022-12-08 21:02:20] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:33016]
frigate    | [2022-12-08 21:02:20] frigate.output                 WARNING : Unable to copy frame camera_11669607701.49046 to birdseye.
frigate    | [2022-12-08 21:02:24] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:33016]
frigate    | [2022-12-08 21:02:43] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:40168]
frigate    | [2022-12-09 09:09:36] frigate.watchdog               INFO    : Detection appears to be stuck. Restarting detection process...
frigate    | [2022-12-09 09:09:36] root                           INFO    : Waiting for detection process to exit gracefully...
frigate    | [2022-12-09 09:10:06] root                           INFO    : Detection process didnt exit. Force killing...
frigate    | [2022-12-09 09:10:06] detector.coral                 INFO    : Starting detection process: 397705
frigate    | [2022-12-09 09:10:06] frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate    | [2022-12-09 09:10:09] frigate.edgetpu                INFO    : TPU found
frigate    | [2022-12-09 09:15:22] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:39030]
frigate    | [2022-12-09 09:16:47] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:39030]
frigate    | [2022-12-09 16:40:41] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:55284]
frigate    | [2022-12-09 16:47:49] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:55284]
[ . . . ]

FFprobe output from your camera

Output for the detect stream of the camera producing above results:


ffprobe version 4.4.2 Copyright (c) 2007-2021 the FFmpeg developers
  built with Apple clang version 14.0.0 (clang-1400.0.29.202)
  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-librsvg --enable-libtheora --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libsoxr --enable-libspeex --enable-libass --enable-libbluray --enable-libzimg --enable-libzvbi --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --enable-zlib --disable-libjack --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-indev=jack --enable-opencl --disable-outdev=xv --enable-audiotoolbox --enable-videotoolbox --enable-sdl2 --disable-securetransport --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --enable-libdav1d --enable-libaom --enable-librav1e --enable-libsvtav1 --arch=x86_64 --enable-x86asm --enable-gpl --enable-postproc --enable-libx264 --enable-libx265 --enable-libxvid --enable-libvidstab --enable-nonfree --enable-libfdk-aac
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from 'rtsp://<delete>:<deleted>@192.168.115.181:554/stream0':
  Metadata:
    title           : RTSP/RTP stream from anjvision ipcamera
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuvj420p(pc, progressive), 1920x1080, 15 tbr, 90k tbn, 180k tbc
  Stream #0:1: Audio: pcm_mulaw, 8000 Hz, 1 channels, s16, 64 kb/s

Frigate stats

No response

Operating system

Other Linux

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

MC200E2_AF

Any other information that may be helpful

Running on Ubuntu 22.04.1 LTS CPU: Intel(R) Core(TM) i3-4130T CPU @ 2.90GHz

[NickM-27]

There are many different tools for this, recommend reading this https://docs.frigate.video/guides/false_positives

The answer depends on the false positive, relative size and shape to a true positive, and if it has a consistent position.

But options include min & max_area, min & max_ratio, object masks, and zones with additional object filters (a car won't be in the yard, for example), adjusting min_score & threshold, etc

After some filtering I've not had a single false positive in many months

[pushpak1990]

I had a similar issue. The culprit in my case was changing the orientation of the camera. This caused cars usually to not be proper rectangles in the image. I use the EfficientDet-Lite3x* model now. This has substantially reduced the FP rate for me at the cost of a slightly worse performance.

[asemev]

There are many different tools for this, recommend reading this https://docs.frigate.video/guides/false_positives

The answer depends on the false positive, relative size and shape to a true positive, and if it has a consistent position.

But options include min & max_area, min & max_ratio, object masks, and zones with additional object filters (a car won't be in the yard, for example), adjusting min_score & threshold, etc

After some filtering I've not had a single false positive in many months

Hi, thanks for this. i read the documentation in that link but just dont know how to calculate the size and ratio of a detection. Could you please give me a few hints on how to do this? many thanks.

So here is whats happening:

1. It keeps thinking that the grill or the umbrellas or the water gun on the floor of the terrace as persons:

And here is me with 84% person detection:

objects:
  track:
    - person
    - cat
  filters:
    person:
      # Optional: minimum width*height of the bounding box for the detected object (default: 0)
      min_area: 5000
      # Optional: maximum width*height of the bounding box for the detected object (default: 24000000)
      max_area: 100000
      # Optional: minimum width/height of the bounding box for the detected object (default: 0)
      min_ratio: 0.5
      # Optional: maximum width/height of the bounding box for the detected object (default: 24000000)
      max_ratio: 2.0
      # Optional: minimum score for the object to initiate tracking (default: shown below)
      min_score: 0.6
      # Optional: minimum decimal percentage for tracked object's computed score to be considered a true positive (default: shown below)
      threshold: 0.85
      # Optional: mask to prevent this object type from being detected in certain areas (default: no mask)
      # Checks based on the bottom center of the bounding box of the object

So how can i determine the min/max area and min/max ratio to eliminate these false positives?

Many thanks!

[NickM-27]

If you look at a snapshot or the debug view the area of the box is shown to the right of the score. That'll give you an idea. The radio is width over height so 0.5 is a tall skinny rectangle, 1 is a square, 2 is a wide short rectangle.

[asemev]

If you look at a snapshot or the debug view the area of the box is shown to the right of the score. That'll give you an idea. The radio is width over height so 0.5 is a tall skinny rectangle, 1 is a square, 2 is a wide short rectangle.

Thank you Nick!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.