[Detector Support]: Detections missing after switching from mobiledet to yolov9

[derhappy]

Describe the problem you are having

I switched from mobiledet using a coral to the base yolov9 model (d88...). Mostly, because the mobiledet confused cats and dogs all the time and our garden parasol base was pretty confidently detected as cat all the time.
Right after switching to yolov9 I was pretty happy about much less false positives. However, I realized pretty soon that I was missing a lot of detections. Initially I turned the min_score for dogs up to .7 like supposed, but even after turning it down to .6 I'm pretty sure I'm still missing detections.
Even worse are cats. With 0.65 on one camera and default settings on all other cams I did not get a single cat detection in the last two weeks. Even when a pretty big cat was walking slowly right in front of the camera in good lighting conditions.
At this point I'm not sure if I can trust the model at all, especially if it will detect people in bad lighting..

Does someone have similar issues? Is there any way I can run multiple models in parallel to compare detections?

Version

0.16.1-e664cb2

Frigate config file

ffmpeg:
  global_args: -hide_banner -loglevel warning
  hwaccel_args: preset-intel-qsv-h264
  input_args: preset-rtsp-generic
#  output_args:
#    preset-record-generic
    # Optional: output args for detect streams (default: shown below)
    #detect: -f rawvideo -pix_fmt yuv420p
    # Optional: output args for record streams (default: shown below)
#    record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an

go2rtc:
  streams:
    gartenseite:
      - rtsp://sadrffsadf/Streaming/Channels/101/
    gartenseite_sub:
      - rtsp://sadrffsadf/Streaming/Channels/102/
    terrasse:
      - rtsp://sadrffsadf/Streaming/Channels/101/
    terrasse_sub:
      - rtsp://sadrffsadf/Streaming/Channels/102/
    gartenende:
      - ffmpeg:gartenseite#video=h264#raw=-vf crop=1280:720:2580:0
    eingang:
      - rtsp://sadrffsadf/axis-media/media.amp?streamprofile=full264&camera=2
    garage:
      - rtsp://sadrffsadf/Streaming/Channels/101/
    eingang_sub:
      - rtsp://sadrffsadf/axis-media/media.amp?streamprofile=detect&camera=2
    klingel_ipad:
      - rtsp://sadrffsadf/axis-media/media.amp?camera=3
    innen:
      - rtsp://sadrffsadf/videoMain

  webrtc:
    candidates:
      - 12345:8555
      - stun:8555

mqtt:
  host: 12345
  user: mqtt
  password: mqtt

cameras:
  garage:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garage
          roles:
            - detect
            - record
    detect:
      width: 2688
      height: 1520
      fps: 5
      stationary:
      # Optional: Frequency for confirming stationary objects (default: shown below)
      # When set to 0, object detection will not confirm stationary objects until movement is detected.
      # If set to 10, object detection will run to confirm the object still exists on every 10th frame.
        interval: 300
        threshold: 100
        max_frames:
          objects:
            person: 300
    motion:
      mask: 
        0.243,0.128,0.765,0.201,0.904,0.394,0.918,0,0,0,0,0.352,0.13,0.191,0.168,0.179
      threshold: 40
      contour_area: 60
      improve_contrast: true
      lightning_threshold: 0.3
    zones:
      rasen:
        coordinates: 
          0.906,0.328,0.775,0.209,0.35,0.159,0.266,0.243,0.692,0.366,0.894,0.403
        inertia: 3
        loitering_time: 0
      einfahrt:
        coordinates: 
          0.234,1,0.805,1,0.903,0.407,0.658,0.363,0.262,0.254,0.045,0.491
        inertia: 3
        loitering_time: 0
    objects:
      track:
        - person
        - car
        - license_plate
        - cat
        - dog
        - bear
        - deer
        - raccoon
        - fox
        - rabbit
        - squirrel
        - badger
        - rodent
        - face
      filters:
        bear:
          mask:
            - 1,1,1,0,0.913,0,0.906,1
            - 1,0,0,0,0,0.113,1,0.139
            - 0,0.295,0,1,0.237,1
            - 0.422,0,0,0,0,0.387
        car:
          min_area: 40000
          mask:
            - 1,1,1,0,0.913,0,0.906,1
            - 0,0.295,0,1,0.237,1
        cat:
          mask:
            - 1,1,1,0,0.913,0,0.906,1
            - 1,0,0,0,0,0.113,1,0.139
            - 0,0.295,0,1,0.237,1
            - 0.422,0,0,0,0,0.387
        dog:
          mask:
            - 1,1,1,0,0.913,0,0.906,1
            - 1,0,0,0,0,0.113,1,0.139
            - 0,0.295,0,1,0.237,1
            - 0.422,0,0,0,0,0.387
        person:
          min_area: 10000
          mask:
            - 526,296,587,79,764,0,0,0,0,581,74,581
    mqtt:
      enabled: true
      timestamp: false
      bounding_box: false
      crop: true
      height: 500
      quality: 90

    review:
      alerts:
        required_zones:
          - einfahrt
          - rasen
      detections:
        required_zones:
          - einfahrt
          - rasen
    ui:
      order: 2

    notifications:
      enabled: false
  gartenseite:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/gartenseite_sub
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/gartenseite
          #input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      width: 1280
      height: 720
      fps: 5
      stationary:
      # Optional: Frequency for confirming stationary objects (default: shown below)
      # When set to 0, object detection will not confirm stationary objects until movement is detected.
      # If set to 10, object detection will run to confirm the object still exists on every 10th frame.
        interval: 10
        max_frames:
          objects:
            person: 300
    motion:
      mask:
        - 329,40,662,21,1188,61,1188,0,0,0,0,135
        - 1280,148,1171,145,1035,650,973,662,962,720,1280,720
      contour_area: 40
      frame_height: 80
    zones:
      tonnen:
        coordinates: 141,656,596,417,439,110,0,272,0,370
      weg:
        coordinates: 1057,202,1127,225,811,720,197,720,153,658,370,624
      gartenseite_all:
        coordinates: 476,104,1216,58,1048,720,0,720,0,271
    objects:
      track:
        - person
        - cat
        - deer
        - raccoon
        - fox
        - rabbit
        - squirrel
        - bird
        - badger
        - rodent
        - face
      filters:
        cat:
          mask:
            - 409,24,1196,61,1188,0,0,0,0,72
            - 1043,636,1196,61,1280,90,1280,720,971,720,968,642
        person:
          min_area: 10000
          mask:
            - 0.32,0.033,0.934,0.085,0.928,0,0,0,0,0.1
            - 0.815,0.883,0.934,0.085,1,0.125,1,1,0.759,1,0.756,0.892
            - 0.192,0.78,0.256,0.923,0.222,0.996,0.145,0.998,0.114,0.891
        deer:
          min_area: 1000
    lpr:
      enabled: false
    mqtt:
      enabled: true
      timestamp: false
      bounding_box: false
      crop: true
      height: 500
      quality: 90
    ui:
      order: 3

    notifications:
      enabled: false
  terrasse:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/terrasse_sub
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/terrasse
          #input_args: preset-rtsp-restream
          roles:
            - record
            - audio
    detect:
      width: 1280
      height: 720
      fps: 5
      stationary:
      # Optional: Frequency for confirming stationary objects (default: shown below)
      # When set to 0, object detection will not confirm stationary objects until movement is detected.
      # If set to 10, object detection will run to confirm the object still exists on every 10th frame.
        interval: 10
        max_frames:
          objects:
            person: 300
    motion:
      mask:
        - 0,0.275,0,0,1,0,1,0.619,0.848,0.513,0.868,0.092,0.292,0.081
        - 0.13,0.415,0.155,0.671,0.215,0.669,0.281,0.578,0.259,0.376,0.199,0.337
      threshold: 30
      contour_area: 30
      improve_contrast: true
    zones:
      terrasse_all:
        coordinates: 0.302,0.083,0.95,0.081,0.923,1,0,1,0,0.376
        inertia: 3
        loitering_time: 0
      pflaster:
        coordinates: 0.92,1,0.947,0.681,0.342,0.418,0.016,0.754,0.09,1
        inertia: 3
        loitering_time: 0
      hinter_garage:
        coordinates: 0.41,0.475,0.548,0.261,0.331,0.226,0.311,0.442
        inertia: 1
        loitering_time: 0
    objects:
      track:
        - person
        - cat
        - raccoon
        - fox
        - rabbit
        - squirrel
        - bird
        - badger
        - rodent
      filters:
        person:
          min_area: 10000
        cat:
          min_score: .65
          threshold: .8
        bird:
          min_score: .75
    lpr:
      enabled: false
    mqtt:
      enabled: true
      timestamp: false
      bounding_box: false
      crop: true
      height: 500
      quality: 90
    ui:
      order: 5

    notifications:
      enabled: false
  innen:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/innen
          roles:
            - detect
            - record
    detect:
      width: 1280
      height: 720
      fps: 5
    zones:
      innen_all:
        coordinates: 0,0,1280,0,1280,720,0,720
    objects:
      track:
        - person
      filters:
        person:
          min_area: 5000
    mqtt:
      enabled: true
      timestamp: false
      bounding_box: false
      crop: true
      height: 500
      quality: 90
    lpr:
      enabled: false
    onvif:
      host: sadrffsadf
      port: 888
      user: 123
      password: 123
    ui:
      order: 6

    motion:
      mask:
        - 0,0,0.182,0,0.184,0.044,0,0.042
        - 0.683,0.201,0.678,0.268,0.688,0.279,0.694,0.207
    notifications:
      enabled: false
  gartenende:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/gartenende
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      width: 1280
      height: 720
      fps: 5
      stationary:
      # Optional: Frequency for confirming stationary objects (default: shown below)
      # When set to 0, object detection will not confirm stationary objects until movement is detected.
      # If set to 10, object detection will run to confirm the object still exists on every 10th frame.
        interval: 10
        max_frames:
          objects:
            person: 300
    record:
      enabled: false
    motion:
      frame_alpha: 0.02
      threshold: 50
      contour_area: 40
      mask: 1,0,1,0.21,0,0.204,0,0
      improve_contrast: true
    objects:
      track:
        - person
      filters:
        person:
          threshold: 0.5
          min_area: 3000
          max_area: 180000
          mask:
            - 0,0,1,0,1,0.213,0,0.204
            - 0.588,0.574,0.598,0.657,0.696,0.696,0.696,0.583,0.644,0.506,0.591,0.481
            - 0.737,0.631,0.747,0.44,0.708,0.446,0.709,0.614
            - 0.741,1,0.756,0.739,0.594,0.675,0.556,0.772,0.549,1
            - 0.295,0.624,0.295,0.478,0.355,0.482,0.366,0.629
            - 0.541,0.825,0.507,0.802,0.422,1,0.541,1,0.557,1
    zones:
      gartenende_all:
        coordinates: 1280,720,1280,0,0,0,0,720
    lpr:
      enabled: false
    mqtt:
      enabled: true
      timestamp: false
      bounding_box: false
      crop: true
      height: 500
      quality: 90
    ui:
      order: 4

    notifications:
      enabled: false
  eingang:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/eingang
          #input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/eingang_sub
          #input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      width: 1920
      height: 1080
      fps: 5
      stationary:
        interval: 50
        max_frames:
          objects:
            person: 300
    zones:
      treppe:
        coordinates: 1502,534,788,650,642,840,1461,925
      einfahrt:
        coordinates: 1462,129,1152,166,979,396,1461,310
      klingel:
        coordinates: 1117,373,1560,175,1467,953,889,883
        objects:
          - person
      eingang_ext:
        coordinates: 0.81,1,0.799,0.043,0.614,0.091,0.207,1
        inertia: 8
        loitering_time: 0
      eingang_all:
        coordinates: 1514,1080,1456,0,1063,0,0,984,0,1080
        objects:
          - car
          - cat
          - bear
          - deer
          - raccoon
          - fox
          - rabbit
          - squirrel
          - package
          - waste_bin
          - badger
          - rodent
    objects:
      track:
        - person
        - car
        - license_plate
        - cat
        - dog
        - bear
        - deer
        - raccoon
        - fox
        - rabbit
        - squirrel
        - package
        - waste_bin
        - badger
        - rodent
        - face
      filters:
        bear:
          mask:
            - 1920,1080,1920,0,1487,0,1511,446,1515,1080
        car:
          min_area: 100000
          mask:
            - 1920,1080,1920,0,1462,0,1485,321,1018,402,495,1080
            - 1268,74,1458,0,1471,246,1279,251
        cat:
          max_area: 50000
          min_score: .65
          threshold: .8
          mask:
            - 1,1,1,0,0.774,0,0.787,0.413,0.789,1
            - 0.43,0,0,0,0,0.604
        dog:
          mask:
            - 1,1,1,0,0.774,0,0.787,0.413,0.789,1
            - 0.43,0,0,0,0,0.604
          min_score: .6
        person:
          mask:
            - 1053,169,1168,115,1182,0,0,0,0,1080,135,1080
    motion:
      mask:
        - 0,0,0.508,0,0.224,0.481,0,0.77
        - 0.316,0.809,0.293,0.856,0.311,0.887,0.341,0.904,0.364,0.895,0.383,0.83,0.372,0.806,0.342,0.776
        - 1,1,0.848,1,0.851,0,1,0
        - 0.236,1,0,0.837,0,1
        - 0.629,1,0.636,0.844,0.499,0.799,0.442,1
        - 0.516,0.576,0.49,0.418,0.537,0.345,0.534,0.276,0.596,0.192,0.668,0.185,0.684,0.237,0.728,0.381,0.705,0.57,0.597,0.595
        - 0.084,1,0.015,0.917,0,0.768,0.055,0.671,0.068,0.775,0.081,0.837,0.096,0.727,0.13,0.735,0.132,0.798,0.15,0.81,0.153,0.839,0.196,0.825,0.234,0.828,0.281,0.869,0.255,1
        - 0.378,1,0.355,0.931,0.28,0.94,0.269,1
      threshold: 35
      contour_area: 25
      improve_contrast: true
    mqtt:
      enabled: true
      timestamp: false
      bounding_box: false
      crop: true
      height: 500
      quality: 95
    review:
      alerts:
        labels:
          - car
          - dog
          - person
        required_zones:
          - einfahrt
          - treppe
          - klingel
          - eingang_ext
      detections:
        required_zones:
          - einfahrt
          - eingang_ext
          - klingel
          - treppe

    ui:
      order: 1
    notifications:
      enabled: false
detectors:
#  coral:
 #   type: edgetpu
#    device: usb
  ov_0:
    type: openvino
    device: GPU
  ov_1:
    type: openvino
    device: GPU
model:
#  path: plus://1234
#default  path: plus://12345
 # path: plus://123451
 #openvino
  path: plus://d88....

record:
  enabled: true
  retain:
    days: 2
    mode: all
  alerts:
    retain:
      days: 8
    pre_capture: 5
    post_capture: 5
  detections:
    retain:
      days: 8
    pre_capture: 5
    post_capture: 5
snapshots:
  enabled: true
  bounding_box: true
  crop: true
  clean_copy: true
  retain:
    default: 8

database:
  path: /db/frigate.db

# Optional: birdseye configuration
birdseye:
  enabled: false
  width: 800
  height: 600
  quality: 20
  mode: continuous

semantic_search:
  enabled: true
  reindex: false

  model_size: small
face_recognition:
  enabled: true
  model_size: large

telemetry:
  version_check: false

version: 0.16-0

notifications:
  enabled: false
  email: sadrffsadf
detect:
  enabled: true
classification:
  bird:
    enabled: false
camera_groups:
  Vorne:
    order: 1
    icon: LuDoorClosed
    cameras:
      - eingang
      - garage
  Hinten:
    order: 2
    icon: LuFlower2
    cameras:
      - gartenseite
      - gartenende
      - terrasse
lpr:
  enabled: false

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true 
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:0.16.1
    shm_size: "364mb" 
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral
      - /dev/dri/renderD128 # for intel hwaccel
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /lightning/lightning/apps/frigate/:/config/
      - /lightning/lightning/apps/frigate:/db
      - /video/video/frigate:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "9050:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
#    network_mode: host
    environment:
      FRIGATE_RTSP_PASSWORD: "1234"
      PLUS_API_KEY: "1234"
    networks:
      - default

networks:
  default:
    name: gateway
    external: true

Relevant Frigate log output

not relevant

Install method

Docker Compose

Object Detector

OpenVino

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[inkeep-ai-assistant[bot]]

Your experience switching from mobiledet (Coral) to a Frigate+ YOLOv9 model is consistent with how these models behave in Frigate. YOLOv9 models are designed for improved accuracy and fewer false positives, but you may encounter missed detections if your object score filters are not tuned for your specific environment(1)(2).

Here are actionable steps to improve detection rates for cats and dogs:

1. Review and Lower Your min_score and threshold Settings

   • For cat and dog, try lowering min_score and threshold further if you are missing detections. For example:

     filters:
       cat:
         min_score: 0.5
         threshold: 0.7
       dog:
         min_score: 0.5
         threshold: 0.7

   • Lowering these values increases the chance to detect less confident objects, but it may also slightly increase false positives. Tune incrementally(2).

2. Submit True and False Positives to Frigate+

   • The Frigate+ model improves as you submit more verified true and false positives (especially for your actual environment and lighting). Make sure you are verifying images in the Frigate+ portal, not just marking them in the Frigate UI(1).

3. Provide Diverse Training Images

   • For best results, submit at least 100 verified images per camera, covering different lighting and seasonal conditions. This helps the model adapt to your specific scenes and reduces missed detections(1).

4. Check Mask Settings

   • Overly aggressive masks (object or motion) can prevent detections in certain regions. Double-check that your masks aren’t unintentionally blocking areas where you expect detections(2).

5. Model Size and Hardware

   • For OpenVINO, using the s size YOLOv9 model at 320x320 is recommended for performance and accuracy. Verify that your model path matches what you’ve tested for your hardware(1).

Running Multiple Models in Parallel:
Frigate does not support running multiple object detection models simultaneously on the same camera feed. You must choose one model per detector(3).

Summary:

• Lower min_score and threshold for missed objects.
• Submit more verified images to Frigate+ for better fine-tuning.
• Check your mask/filter settings.
• You cannot run two detection models in parallel for the same stream.

For further model tuning details and hardware compatibility, see the Frigate+ Models documentation and Object Filters documentation.

📚 Sources:

1. Models
2. Filters
3. Supported Hardware

---

Have another question? Just tag @inkeep.

[NickM-27]

If you're missing detections the first thing you should do is remove your area filters, this is an easy way to miss things without realizing it since they just get filtered out.

You're also running at a very high detect resolution and I think setting the detect resolution to 1280x720 to have the GPU scale it down will work better

[derhappy]

Thanks for your quick reply Nick!
The areas should not be an issue since I did not change those when switching the model. It's mostly greenery and walls where no relevant object can go / the bounding box should never hit.
When initially setting up frigate I intentionally used the high resolutions on the "eingang" and "garage" cameras since I was missing object further away from the cameras. I'll give it another try with the new model.