[Detector Support]: ONXX and OpenVINO

[alsonh]

Describe the problem you are having

From the documentation, it appears that the ONXX detector will use OpenVINO when detected. I've configured my system to use the YOLOv9 model, but have noticed a big difference in CPU usage when using ONXX vs OpenVINO. With OpenVINO, it's averaging only 1-2% CPU usage, whereas with ONXX it spikes over 300% at times. I'm running Intel UHD Graphics 630.

What is the difference between using the openvino detector vs onxx detector (that uses OpenVINO anyway)?

Version

0.16.1-e664cb2

Frigate config file

mqtt:
  enabled: true
  host: <REDACTED_IP>
  user: <REDACTED_USER>
  password: <REDACTED_PASSWORD>

detectors:
  ov:
    type: openvino
    device: GPU
#  onnx:
#    type: onnx

model:
  model_type: yolo-generic
  width: 320
  height: 320
  input_tensor: nchw
  input_dtype: float
  path: /config/model_cache/yolov9-t-320.onnx
  labelmap_path: /labelmap/coco-80.txt

cameras:
  Living_Room:
    ffmpeg:
      inputs:
        - path: rtsp://<REDACTED_USER>:<REDACTED_PASSWORD>@<REDACTED_IP>/stream2
          roles:
            - detect
        - path: rtsp://<REDACTED_USER>:<REDACTED_PASSWORD>@<REDACTED_IP>/stream1
          roles:
            - record
      hwaccel_args: preset-vaapi
    detect:
      enabled: true
    record:
      enabled: true
    snapshots:
      enabled: true
    motion:
      threshold: 50
    lpr:
      enabled: False

  Front_door_1:
    ffmpeg:
      inputs:
        - path: rtsp://<REDACTED_USER>:<REDACTED_PASSWORD>@<REDACTED_IP>/cam/realmonitor?channel=1&subtype=0
          roles:
            - record
        - path: rtsp://<REDACTED_USER>:<REDACTED_PASSWORD>@<REDACTED_IP>/cam/realmonitor?channel=1&subtype=2
          roles:
            - detect
      hwaccel_args: preset-vaapi
    detect:
      enabled: true
    record:
      enabled: true
    snapshots:
      enabled: true
    motion:
      threshold: 50
    lpr:
      enabled: False

  Front_door_2:
    ffmpeg:
      inputs:
        - path: rtsp://<REDACTED_USER>:<REDACTED_PASSWORD>@<REDACTED_IP>/cam/realmonitor?channel=2&subtype=0
          roles:
            - record
        - path: rtsp://<REDACTED_USER>:<REDACTED_PASSWORD>@<REDACTED_IP>/cam/realmonitor?channel=2&subtype=2
          roles:
            - detect
      hwaccel_args: preset-vaapi
    detect:
      enabled: true
    record:
      enabled: true
    snapshots:
      enabled: true
    motion:
      threshold: 50
    lpr:
      enabled: True

record:
  enabled: true
  retain:
    days: 7
    mode: motion
  alerts:
    retain:
      days: 30
  detections:
    retain:
      days: 30

face_recognition:
  enabled: true
  model_size: large

lpr:
  enabled: True

objects:
  track:
    - person
    - car

version: 0.16-0
detect:
  enabled: true
semantic_search:
  enabled: false
  model_size: small
classification:
  bird:
    enabled: false

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "138mb"
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/frigate/config:/config
      - /mnt/media:/media/frigate
#      - nfsvolume:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "8971:8971"
      - "5000:5000"
      - "8554:8554"
      - "8555:8555/tcp"
      - "8555:8555/udp"
      - "1984:1984"
    environment:
      FRIGATE_RTSP_PASSWORD: "<REDACTED_PASSWORD>"

Relevant Frigate log output

N/A

Install method

Docker Compose

Object Detector

OpenVino

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[inkeep-ai-assistant[bot]]

The difference comes down to how Frigate routes detection workloads and which backend is actually doing the inference:

• When you specify type: openvino, Frigate directly uses the OpenVINO backend for object detection. This is the recommended way to leverage Intel hardware acceleration, especially on Intel CPUs and GPUs(1).
• When you specify type: onnx, Frigate runs the ONNX detector. On Intel hardware, if the default Frigate image is used and a supported ONNX model is loaded, OpenVINO will be automatically detected and used as the backend for ONNX inference(1)(2). However, this automatic routing may not be as efficient as using the OpenVINO detector directly.

Directly configuring the OpenVINO detector (type: openvino) ensures Frigate uses the most optimized pipeline for Intel hardware, which typically results in lower CPU usage and better performance. The ONNX detector is designed for broader compatibility and may introduce additional overhead, especially if it is not leveraging the hardware as efficiently(1)(2).

In summary, for Intel UHD Graphics 630, using the OpenVINO detector is the most efficient and recommended configuration for object detection in Frigate(1)(3)(4)(2).

📚 Sources:

1. Supported Hardware
2. Supported Hardware
3. Recommended hardware
4. Supported Hardware

---

Have another question? Just tag @inkeep.

[NickM-27]

The onnx detector runs on CPU because there's a bug that causes openvino to not be used. In general it's recommended to just use OpenVINO detector

[MYeager1967]

I'm trying to get this running with the onnx detector. I managed to get it running but it's not using my Invidia GPU and is HORRIBLE on CPU....

Is there a trick to getting it to use the GPU? It shows up in the system stats, it's just not being used...

[NickM-27]

You need to create your own support discussion, but you need to make sure that it is using the -tensorrt docker image. That is all that is needed assuming GPU is setup correctly