Openvino device:GPU works again and is 200x more cpu efficient

[Reverse256]

my CPU: Intel Ultra 5 125H

openvino "device:GPU" was unusable for many a while back, so I used CPU as suggested.

My cpu usage would be 700%-1200% (rarely 1700%)
inference 25ms

With openvino "device: GPU"

320x320 frigate+ model

640x640 frigate+ model

Thank you developers <3

[TheSpaceGod]

Mine still seems to be using the CPU for an Intel i5 6500T in a Dell Optiplex. :/
Any ideas on why that might be?

###############################################################################
#https://docs.frigate.video/configuration/hardware_acceleration/#intel-based-cpus
ffmpeg:
  hwaccel_args: auto

###############################################################################
#https://docs.frigate.video/configuration/object_detectors/#openvino-detector
detectors:
  ov:
    type: openvino
    device: GPU

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  path: /openvino-model/ssdlite_mobilenet_v2.xml
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

###############################################################################

[Reverse256]

Are you running virtualization on your 125H? If so, what did you have to do to get it working? I cant get my 185H to work running vmware + docker + ubuntu 24.04.

I'm the OP with the 125H, docker. Back when I first got this, I had to run a newer kernel for the GPU. I am currently on Linux kernel 6.12.12

This is with 3 cameras running BTW, but im pretty sure its using a CPU detector and not a GPU detector based on overall CPU utilization.

Could be using more than me if your cameras are 4k. Mine are 1080p.
Strange, I have no processes named 'addon.' They're all descriptive for my docker setup: "detector," "image embed," or "ffmpeg."

Your inference must be running on the GPU I think. My much newer CPU inference was 25ms. Yours is 10ms like my intel igpu.

[shjips]

Are you running virtualization on your 125H? If so, what did you have to do to get it working? I cant get my 185H to work running vmware + docker + ubuntu 24.04.

I'm the OP with the 125H, docker. Back when I first got this, I had to run a newer kernel for the GPU. I am currently on Linux kernel 6.12.12

Are you also running a hypervisor? Or bare metal Ubuntu? Im running vmware + ubuntu as a VM and docker on it. Ubuntu kernel 6.12.19 now, but still have issues.
Can you maybe post your ffmpeg conf + docker conf?

[TheSpaceGod]

@Reverse256, my docker container's are prefixed with "addon" because they are running as home assistant "addons" on home assistant OS.
One of my cameras is 4k, maybe that is it. I would just think if I am using the GPU for transcode and detection, that the CPU should not be used for much.

[Reverse256]

@Reverse256, my docker container's are prefixed with "addon" because they are running as home assistant "addons" on home assistant OS. One of my cameras is 4k, maybe that is it. I would just think if I am using the GPU for transcode and detection, that the CPU should not be used for much.

Oh, I see. The GPU can be used to decode the video stream and detect for an object. It uses the CPU to view all pixels and calculate if the pixels change colors change enough which would denote detected movement. After movement is detected then it sends that area of the video frame to the object detector (now the GPU).

might be explained better here in the docs

[Reverse256]

Are you also running a hypervisor? Or bare metal Ubuntu? Im running vmware + ubuntu as a VM and docker on it. Ubuntu kernel 6.12.19 now, but still have issues. Can you maybe post your ffmpeg conf + docker conf?

bare metal manjaro + docker for frigate

I don't think I have an ffmpeg conf. If you know what file it is, I can post it. All of my config I believe is here in the frigate config:

mqtt:
  # Optional: Enable publishing snapshot via mqtt for camera (default: shown below)
  # NOTE: Only applies to publishing image data to MQTT via 'frigate/<camera_name>/<object_name>/snapshot'.
  # All other messages will still be published.
  enabled: true
  host: 10.1.2.6
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: user
  password: password

semantic_search:
  enabled: true
  reindex: false
  model_size: large

cameras:
#  dummy_camera: # <--- this will be changed to your actual camera later
#    enabled: False
#    ffmpeg:
#      inputs:
#        - path: rtsp://127.0.0.1:554/rtsp
#          roles:
#            - detect
  Yard:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Yard
          input_args: preset-rtsp-restream
          roles:
            - audio
            - record
            - detect
  #      - path: rtsp://127.0.0.1:8554/Yard_Sub
  #        input_args: preset-rtsp-restream
 #         roles:
 #           - detect 
    zones:
      yard:
        coordinates: 
          0,1,1,1,1,0.388,1,0.264,0.93,0.222,0.917,0.377,0.742,0.371,0.629,0.353,0.269,0.373,0,0.403
        inertia: 3
        loitering_time: 0
      sidewalk:
        coordinates: 
          0,0.252,0.501,0.2,0.925,0.272,0.918,0.38,0.894,0.379,0.746,0.367,0.539,0.358,0.28,0.373,0,0.4
        loitering_time: 0
        inertia: 3
    snapshots:
      required_zones:
        - yard
        - sidewalk
    review:
      alerts:
        required_zones:
          - yard
      detections:
        required_zones:
          - sidewalk
    motion:
      mask: 0.92,0,0.918,0.085,1,0.167,1,0
    objects: {}
    detect:
      annotation_offset: -700
  Garage:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Garage
          input_args: preset-rtsp-restream
          roles:
            - audio
            - record
            - detect
     #   - path: rtsp://127.0.0.1:8554/Garage_Sub
     #     input_args: preset-rtsp-restream
     #     roles:
    #        - detect
    zones:
      driveway:
        coordinates: 1,0.526,1,1,0,1,0,0.372,0.284,0.373,0.586,0.384,0.793,0.539
        inertia: 3
        loitering_time: 0
      yard:
        coordinates: 
          0.586,0.384,0.792,0.542,1,0.53,1,0.318,0.975,0.317,0.943,0.413,0.836,0.408,0.757,0.399
        loitering_time: 0
        inertia: 3
      sidewalk:
        coordinates: 
          0.973,0.32,0.944,0.414,0.814,0.407,0.621,0.385,0.23,0.371,0.096,0.371,0,0.372,0.001,0.324,0.605,0.359,0.845,0.38,0.919,0.388,0.945,0.308
        loitering_time: 0
        inertia: 3
    review:
      alerts:
        required_zones:
          - driveway
          - yard
      detections:
        required_zones:
          - sidewalk
    snapshots:
      required_zones:
        - driveway
        - yard
        - sidewalk
#  Door:
#    enabled: True
#    ffmpeg:
#      inputs:
#       # - path: rtsp://ghfkvl0001:[email protected]:554/mpeg/media.amp
#        - path: rtsp://127.0.0.1:8554/Door
#          input_args: preset-rtsp-restream
#          roles:
#            - audio
        #    - record
   #     - path: rtsp://127.0.0.1:8554/Garage_Sub
   #       input_args: preset-rtsp-restream
   #       roles:
   #         - detect
   # zones:
   #   All:
   #     coordinates: 640,480,0,480,0,183,640,188
   # record:
   #   events:
   #     required_zones:
   #       - All
   # snapshots:
   #   required_zones:
   #     - All
    motion:
      mask:
        - 0,0,0,0.196,0.113,0.179,0.142,0.13,1,0.22,1,0
        - 0.918,0.252,1,0.269,1,0.322,0.911,0.305
    objects:
      mask: 0.95,0.455,0.919,0.464,0.925,0.515
    detect:
      annotation_offset: -970
objects:
  track:
    - person
    - face
    - license_plate
    - dog
    - cat
#    - car
    - amazon
    - fedex
    - ups
    - package
    - deer

audio:
  enabled: true
  # Optional: Configure the amount of seconds without detected audio to end the event (default: 30)
  max_not_heard: 30
  # Optional: Configure the min rms volume required to run audio detection
  # As a rule of thumb:
  #  - 200 - high sensitivity
  #  - 500 - medium sensitivity (default)
  #  - 1000 - low sensitivity
  min_volume: 500
  # Optional: Types of audio to listen for (default: bark, fire_alarm, scream, speech, yell)
  listen:
    - fire_alarm
    - scream
    - speech
    - yell
  # Optional: Filters to configure detection.
  filters:
    # Label that matches label in listen config.
    speech:
      # Minimum score that triggers an audio event (default: shown below)
      threshold: 0.8
    scream:
      threshold: 0.6
    yell:
      threshold: 0.6

go2rtc:
  streams:
    Yard:
      - rtsp://admin:[email protected]:554/Streaming/Channels/101
    #  - "ffmpeg:Yard#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc
    Yard_Sub:
      - rtsp://admin:[email protected]:554/Streaming/Channels/102
    #  - "ffmpeg:Yard#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc
    Garage:
      - rtsp://admin:[email protected]:554/Streaming/Channels/101
    #  - "ffmpeg:Garage#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc
    Garage_Sub:
      - rtsp://admin:[email protected]:554/Streaming/Channels/102
    #  - "ffmpeg:Garage#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc
#    Door:
#      - rtsp://ghfkvl0001:[email protected]:554/mpeg/media.amp
#      - "ffmpeg:Garage#audio=opus" # <- copy of the stream which transcodes audio to opus for webrtc

  webrtc:
    candidates:
      - 10.1.2.45:8555
      - stun:8555

record:
############################## True if record ######################################
  enabled: true
  sync_recordings: true
  retain:
    days: 9
    mode: all
  alerts:
    retain:
      days: 90
  detections:
    retain:
      days: 90
snapshots:
  enabled: true
  retain:
    default: 30


birdseye:
  enabled: true
  mode: continuous

ffmpeg:
  hwaccel_args: preset-vaapi
  output_args:
    record: preset-record-generic-audio-copy
#  hwaccel_args: preset-nvidia-h264

detectors:
  ov:
    type: openvino
    device: GPU
#  tensorrt:
#    type: tensorrt
#    device: 0 #This is the default, select the first GPU


###### temp intel GPU model while no custom model for gpu ##########
#model:
#  width: 300
#  height: 300
#  input_tensor: nhwc
#  input_pixel_format: bgr
#  path: /openvino-model/ssdlite_mobilenet_v2.xml
#  labelmap_path: /openvino-model/coco_91cl_bkgr.txt


model:
  path: plus://5195ff53a5e86a3813d8c111ee809963

#  input_tensor: nchw
#  input_pixel_format: rgb
#  width: 640
#  height: 640
version: 0.16-0
camera_groups: {}

my docker compose:

services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    stop_grace_period: 30s
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:cf16eda-amd64
    shm_size: "700mb" ## 3 cameras: 1080p=115MB; 4k=375MB
    networks:
      frigate_NET:
        ipv4_address: 172.19.0.2

    devices:
#      - /dev/bus/usb:/dev/bus/usb  # Passes the USB Coral, needs to be modified for other versions
#      - /dev/video19:/dev/video19
#      - /dev/video20:/dev/video20
#      - /dev/video21:/dev/video21
#      - /dev/video22:/dev/video22
#      - /dev/video23:/dev/video23
#      - /dev/video24:/dev/video24
#      - /dev/video25:/dev/video25
#      - /dev/video26:/dev/video26
#      - /dev/video27:/dev/video27
#      - /dev/video28:/dev/video28
#      - /dev/video29:/dev/video29
#      - /dev/video30:/dev/video30
#      - /dev/video31:/dev/video31
#      - /dev/video32:/dev/video32
#      - /dev/video33:/dev/video33
#      - /dev/video34:/dev/video34
#      - /dev/video35:/dev/video35
#      - /dev/video36:/dev/video36
#      - /dev/apex_0:/dev/apex_0    # Passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
#      - /dev/video11:/dev/video11  # For Raspberry Pi 4B
      - /dev/dri/renderD128:/dev/dri/renderD128 # For intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
      - ./frigate/config:/config
      - /home/user/Videos/camera:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1.8g
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      - PLUS_API_KEY=aaaaaaaaa
      - FRIGATE_RTSP_PASSWORD:"password"
      - TZ=aaaaa/aaaaaa

networks:
  frigate_NET:
    driver: bridge
    ipam:
      config:
        - subnet: 172.19.0.0/16
          gateway: 172.19.0.1
          ip_range: 172.19.0.0/24