Best model for CPU-based person detector?

[Largelos]

Describe the problem you are having

I tried setting up Frigate on a server without hardware acceleration (Ryzen 7 5950x), but I ran into an issue where people are not always detected. I often see situations where a person is detected by indoor cameras, but the entry/exit event on the outdoor camera is not registered for them. I suspect the reason might be that a person can appear in the frame and leave before they are recognized, although I’m not completely sure. Troubleshooting this is difficult because these situations are usually analyzed after the fact, and the event history records doesn’t include any debug data like motion detection and objects stats. However, when reviewing recordings, I sometimes notice that a person appears in the frame for about 5 seconds before their snapshot is actually taken. I suspect the core issue is insufficient detection accuracy, and that the system doesn’t collect enough frames with a detection confidence above default 0.7 value.

So my question is: does the Frigate+ paid subscription provide CPU-friendly models that don’t require a lot of computational power, compared to free one, but offer better person detection accuracy? I’ve tried the default CPU and /openvino-model/ssdlite_mobilenet_v2.xml, but I didn’t notice any difference. Or mb any other free but better models exist for CPU processing?

Or maybe there is a way to record debug information into the history so I can understand what exactly is happening in those moments?

Version

0.16.2-4d58206

Frigate config file

mqtt:
  enabled: false
go2rtc:
  streams:
    cam1_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/101
    cam1_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/102

    cam2_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/201
    cam2_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/202

    cam3_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/301
    cam3_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/302

    cam4_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/401
    cam4_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/402

    cam5_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/501
    cam5_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/502

    cam6_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/601
    cam6_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/602

    cam7_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/701
    cam7_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/702

    cam8_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/801
    cam8_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/802

    cam9_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/901
    cam9_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/902

    cam10_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/1001
    cam10_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/1002

    cam11_main:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/1101
    cam11_sub:
      - rtsp://admin:<pass>@10.16.2.27:554/Streaming/Channels/1102

cameras:
  cam1:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam1_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
      stationary:
        interval: 50
        threshold: 25
    live:
      streams:
        Main Stream: cam1_main
        Sub Stream: cam1_sub
    motion:
      mask: 0.067,0.097,0.578,0.18,0.835,0.313,0.83,1,1,1,1,0,0,0,0,0.125
      threshold: 40
      contour_area: 10
      improve_contrast: true
    zones:
      Near_office:
        coordinates: 0.002,0.229,0.397,0.26,0.826,0.356,0.83,0.999,0,1
        loitering_time: 0
      Office:
        coordinates: 0.72,0.996,0.778,0.603,0.547,0.591,0.42,0.997
        inertia: 1
        loitering_time: 0
        objects: person
    objects:
      filters:
        car:
          mask: 0.026,0.098,0.028,0.177,0.15,0.169,0.158,0.091
    record:
      enabled: true
      detections:
        pre_capture: 5
        post_capture: 5
        retain:
          days: 14
          mode: active_objects
  cam2:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam2_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
    live:
      streams:
        Main Stream: cam2_main
        Sub Stream: cam2_sub
    motion:
      mask: 0,0,0,0.975,0.165,0.245,0.316,0.047
    zones:
      Office:
        coordinates: 
          0.224,1,0.112,0.499,0.187,0.241,0.384,0.172,0.645,0.144,1,0.572,1,1
        inertia: 1
        loitering_time: 0
        objects: person
  cam3:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam3_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
    live:
      streams:
        Main Stream: cam3_main
        Sub Stream: cam3_sub

    motion:
      mask: 
        0,0,0,0.178,0.149,0.086,0.286,0.086,0.287,0.049,0.255,0.052,0.255,0.005
    zones:
      Office:
        coordinates: 0.035,0.978,0.109,0.446,0.502,0.174,0.82,0.205,0.852,0.997
        inertia: 1
        loitering_time: 0
        objects: person
  cam4:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam4_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
    live:
      streams:
        Main Stream: cam4_main
        Sub Stream: cam4_sub

    motion:
      mask: 0.043,0.088,0.292,0.085,0.293,0.049,0.043,0.048
    zones:
      Office:
        coordinates: 1,1,1,0.564,0.434,0.19,0.058,0.232,0.001,0.49,0.006,0.996
        inertia: 1
        loitering_time: 0
        objects: person
  cam5:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam5_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 720
      height: 1280
      stationary:
        interval: 50
        threshold: 25
    live:
      streams:
        Main Stream: cam5_main
        Sub Stream: cam5_sub
    motion:
      mask: 0.004,0.285,0.327,0.205,1,0.191,1,0,0.004,0
      threshold: 40
      contour_area: 10
      improve_contrast: true
    zones:
      Near_office:
        coordinates: 1,1,0.992,0.218,0.363,0.237,0,0.387,0.005,0.996
        loitering_time: 0
    record:
      enabled: true
      detections:
        pre_capture: 5
        post_capture: 5
        retain:
          days: 14
          mode: active_objects
  cam6:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam6_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
      stationary:
        interval: 50
        threshold: 25
    live:
      streams:
        Main Stream: cam6_main
        Sub Stream: cam6_sub
    motion:
      mask: 0,0.22,0.336,0.112,1,0.316,1,0,0,0
      threshold: 40
      contour_area: 10
      improve_contrast: true
    objects:
      filters:
        car:
          mask: 0,0.202,0.379,0.108,0.606,0.163,1,0.313,1,0,0,0
    zones:
      Near_office:
        coordinates: 0.215,1,0.083,0.261,0.38,0.138,1,0.34,1,1
        loitering_time: 0
    record:
      enabled: true
      detections:
        pre_capture: 5
        post_capture: 5
        retain:
          days: 14
          mode: active_objects
  cam7:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam7_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
    live:
      streams:
        Main Stream: cam7_main
        Sub Stream: cam7_sub

    motion:
      mask: 0.036,0.051,0.039,0.09,0.291,0.091,0.292,0.056
    zones:
      Near_office:
        coordinates: 0.004,0.996,0,0.409,0.245,0,1,0,1,1
        loitering_time: 0
  cam8:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam8_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
      fps: 10
    live:
      streams:
        Main Stream: cam8_main
        Sub Stream: cam8_sub

    motion:
      mask: 0.032,0.048,0.036,0.101,0.294,0.098,0.294,0.043
    zones:
      Office:
        coordinates: 0.059,0.597,0.486,0.335,0.873,0.569,0.812,0.996,0.05,0.99
        inertia: 1
        loitering_time: 0
        objects: person
  cam9:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam9_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
    live:
      streams:
        Main Stream: cam9_main
        Sub Stream: cam9_sub

    motion:
      mask: 0.039,0.048,0.037,0.093,0.293,0.091,0.29,0.045
    zones:
      Office:
        coordinates: 0.106,0.344,0.515,0.268,0.999,0.647,0.998,0.996,0.03,1
        inertia: 1
        loitering_time: 0
        objects: person
  cam10:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam10_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
    live:
      streams:
        Main Stream: cam10_main
        Sub Stream: cam10_sub

    motion:
      mask: 0.266,0.094,0.176,0.096,0.176,0.057,0.266,0.052
    zones:
      Office:
        coordinates: 0,1,0.035,0.148,0.986,0.111,0.978,0.991
        inertia: 1
        loitering_time: 0
        objects: person
  cam11:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/cam11_main
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
    live:
      streams:
        Main Stream: cam11_main
        Sub Stream: cam11_sub

    zones:
      Office:
        coordinates: 0.236,0.991,0.804,0.991,0.982,0.01,0.082,0.012
        inertia: 1
        loitering_time: 0
        objects: person
version: 0.16-0

snapshots:
  enabled: true
  clean_copy: true

detect:
  enabled: true
detectors:
  ov:
    type: openvino
    device: CPU
model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  path: /openvino-model/ssdlite_mobilenet_v2.xml
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt
review:
  # Optional: alerts configuration
  alerts:
    # Optional: enables alerts for the camera (default: shown below)
    enabled: true
    # Optional: labels that qualify as an alert (default: shown below)
    labels:
      - person
    # Optional: required zones for an object to be marked as an alert (default: none)
    # NOTE: when settings required zones globally, this zone must exist on all cameras
    #       or the config will be considered invalid. In that case the required_zones
    #       should be configured at the camera level.
    #required_zones:
    #  - driveway
  # Optional: detections configuration
  detections:
    # Optional: enables detections for the camera (default: shown below)
    enabled: true
    # Optional: labels that qualify as a detection (default: all labels that are tracked / listened to)
    labels:
      - car
      - person
    # Optional: required zones for an object to be marked as a detection (default: none)
    # NOTE: when settings required zones globally, this zone must exist on all cameras
    #       or the config will be considered invalid. In that case the required_zones
    #       should be configured at the camera level.
    #required_zones:
    #  - driveway
objects:
  track:
    - person
    - car
record:
  enabled: true
  #sync_recordings: true
  detections:
    # Optional: Number of seconds before the detection to include (default: shown below)
    pre_capture: 5
    # Optional: Number of seconds after the detection to include (default: shown below)
    post_capture: 5
    # Optional: Retention settings for recordings of detections
    retain:
      # Required: Retention days (default: shown below)
      days: 14
      # Optional: Mode for retention. (default: shown below)
      #   all - save all recording segments for detections regardless of activity
      #   motion - save all recordings segments for detections with any detected motion
      #   active_objects - save all recording segments for detections with active/moving objects
      #
      # NOTE: If the retain mode for the camera is more restrictive than the mode configured
      #       here, the segments will already be gone by the time this mode is applied.
      #       For example, if the camera retain mode is "motion", the segments without motion are
      #       never stored, so setting the mode to "all" here won't bring them back.
      mode: motion
camera_groups: {}

docker-compose file or Docker CLI command

N/A

Relevant Frigate log output

N/A

Install method

Home Assistant Add-on

Object Detector

OpenVino

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[inkeep-ai-assistant[bot]]

For CPU-based person detection, the mobiledet model type is the most CPU-friendly option(1). You're already using the default OpenVINO SSDLite MobileNet v2 model, which is appropriate for CPU usage(2).

Regarding Frigate+ models: Yes, Frigate+ does offer a mobiledet model type that "runs on Google Coral devices and CPUs"(1). This model is "based on the same architecture as the default model included with Frigate"(1). However, it's specifically trained on real security camera footage from the Frigate community, which can provide better accuracy for your specific scenes(1).

For CPU hardware specifically, the documentation recommends: "CPU (not recommended for actual use: Use a CPU to run tflite model, this is not recommended and in most cases OpenVINO can be used in CPU mode with better results"(3). Since you're already using OpenVINO in CPU mode, you're on the right track.

Key recommendations:

1. Consider Frigate+ mobiledet models: These are optimized for CPUs and trained on real security footage, potentially improving accuracy(1). With a Frigate+ subscription, you get "12 model trainings to fine tune your model per year"(1).

2. Tune your detection settings: Instead of just changing models, consider:

   • Lowering your detection threshold slightly below 0.7 to catch more detections
   • Adjusting motion masks to ensure people aren't filtered out
   • Reviewing your zone configurations

3. Hardware consideration: The documentation notes that "OpenVino can run on Intel Arc GPUs, Intel integrated GPUs, and Intel CPUs"(3). Your Ryzen 7 5950x is an AMD CPU, and while OpenVINO "will also run on AMD CPUs despite having no official support for it"(2), you might see better performance with dedicated hardware acceleration.

Regarding debug information: The documentation doesn't describe a built-in way to record detailed frame-by-frame detection statistics to history for post-event analysis. You may need to monitor the debug view in real-time or enable debug logging during specific time periods.

📚 Sources:

1. Models
2. Supported Hardware
3. Supported Hardware

---

Have another question? Just tag @inkeep.