Issue with GPU polling after implementing Cora TPU M.2 for detection

[kodiconnect]

Hello,

I am using an Intel ARC a770 for all gpu needs outside of detection which is handled by a Coral M.2. I'm noticed that Frigate can no longer poll the intel GPU PMU with this hybrid setup but frigate will poll the GPU fine if the Coral device is not configured for detection. I'm new to this so I'm more likely to think it's a me issue than a bug - here is my config:

mqtt:
  host: <mqtt_host>
  port: 1883
  user: '<mqtt_user>'
  password: '<mqtt_password>'
  topic_prefix: frigate
  client_id: frigate

ffmpeg:
  input_args:
    - -rtsp_transport
    - tcp
    - -r
    - '20'
  hwaccel_args:
    - -hwaccel
    - vaapi
    - -hwaccel_device
    - /dev/dri/renderD128
    - -hwaccel_output_format
    - yuv420p

detectors:
  coral:
    type: edgetpu
    device: pci

model:
  width: 320
  height: 320
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt
  path: /edgetpu_model.tflite

cameras:
  camera_1:
    ffmpeg:
      inputs:
        - path: rtsp://<camera_1_rtsp_path>/sub
          roles:
            - detect
        - path: rtsp://<camera_1_rtsp_path>/main
          roles:
            - record
    detect:
      width: 640
      height: 360
      fps: 10
    record:
      enabled: true
      retain:
        days: 7
        mode: all
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 7
    objects:
      track:
        - person
        - car

  camera_2:
    ffmpeg:
      inputs:
        - path: rtsp://<camera_2_rtsp_path>/sub
          roles:
            - detect
        - path: rtsp://<camera_2_rtsp_path>/main
          roles:
            - record
    detect:
      width: 640
      height: 360
      fps: 10
    record:
      enabled: true
      retain:
        days: 7
        mode: all
    snapshots:
      enabled: true
      timestamp: true
      bounding_box: true
      retain:
        default: 7
    objects:
      track:
        - person
        - car

version: 0.14

[NickM-27]

This config is invalid for a few reasons:

1. Trying to use an openvino model with a coral
2. Using manual ffmpeg args instead of preset
3. In general the arc should use qsv with Frigate 0.15

[kodiconnect]

Thank you for the rapid reply! I failed to notice how out of date the Unriads ‘official’ frigate community app is 2022. I started with some of your recommendations but could not get QSV to work seemingly because it’s running .14 . I’m guessing the below closer to what is ‘valid’ assuming I can get either get a custom install of frigate in my docker instance or volunteer to maintain the community app. The crazy thing is that the ‘invalid’ config worked but likely would have been a power consumption disaster along with poorer than acceptable performance. ``` mqtt: host: 192.168.1.27 port: 1883 user: '' password: '' topic_prefix: frigate client_id: frigate ffmpeg: hwaccel_args: preset-qsv # Optimized for Intel Arc GPUs with QSV detectors: coral: type: edgetpu device: pci model: width: 320 height: 320 input_tensor: nhwc input_pixel_format: bgr labelmap_path: /openvino-model/coco_91cl_bkgr.txt path: /edgetpu_model/efficientdet_lite0.tflite cameras: front_driveway_POV_1: ffmpeg: inputs: - path: ***@***.***:554/h265Preview_01_sub roles: - detect - path: ***@***.***:554/h265Preview_01_main roles: - record detect: width: 640 height: 360 fps: 10 record: enabled: true retain: days: 7 mode: all snapshots: enabled: true timestamp: true bounding_box: true retain: default: 7 objects: track: - person - car filters: car: mask: 0.108,0.143,0.927,0.326,0.928,0.252,0.588,0.126,0.232,0.081 front_driveway_POV_2: ffmpeg: inputs: - path: ***@***.***:554/h265Preview_02_sub roles: - detect - path: ***@***.***:554/h265Preview_02_main roles: - record detect: width: 640 height: 360 fps: 10 record: enabled: true retain: days: 7 mode: all snapshots: enabled: true timestamp: true bounding_box: true retain: default: 7 objects: track: - person - car version: 0.15 ```

[NickM-27]

you would just need to change the image in the unraid config, no reason to find another app or make changes there

[kodiconnect]

Thank you, sir. Here is my now working config after implementing your recommendations, please let me know if the core pieces are still invalid, thank you!: mqtt: host: <your_mqtt_host> # Replace with your MQTT host IP port: 1883 user: '' # Add your MQTT user if applicable password: '' # Add your MQTT password if applicable topic_prefix: frigate client_id: frigate ffmpeg: hwaccel_args: preset-vaapi # Optimized for Intel Arc GPU hardware acceleration detectors: coral: type: edgetpu device: pci model: width: 300 height: 300 input_tensor: nhwc input_pixel_format: bgr labelmap_path: /config/models/coco_labels.txt # Correct path for label map path: /config/models/efficientdet_lite0.tflite # Correct path for the Coral TPU model cameras: front_driveway_POV_1: ffmpeg: inputs: - path: rtsp://<user>:<password>@<camera_ip>/h265Preview_01_sub # Replace with your credentials and camera IP roles: - detect - path: rtsp://<user>:<password>@<camera_ip>/h265Preview_01_main # Replace with your credentials and camera IP roles: - record detect: width: 640 height: 360 fps: 10 record: enabled: true retain: days: 7 mode: all snapshots: enabled: true timestamp: true bounding_box: true retain: default: 7 objects: track: - person - car filters: car: mask: 0.108,0.143,0.927,0.326,0.928,0.252,0.588,0.126,0.232,0.081 front_driveway_POV_2: ffmpeg: inputs: - path: rtsp://<user>:<password>@<camera_ip>/h265Preview_02_sub # Replace with your credentials and camera IP roles: - detect - path: rtsp://<user>:<password>@<camera_ip>/h265Preview_02_main # Replace with your credentials and camera IP roles: - record detect: width: 640 height: 360 fps: 10 record: enabled: true retain: days: 7 mode: all snapshots: enabled: true timestamp: true bounding_box: true retain: default: 7 objects: track: - person - car version: 0.15-0 From: Nicolas Mowen ***@***.***> Sent: Friday, January 10, 2025 6:42 AM To: blakeblackshear/frigate ***@***.***> Cc: kodiconnect ***@***.***>; Author ***@***.***> Subject: Re: [blakeblackshear/frigate] Issue with GPU polling after implementing Cora TPU M.2 for detection (Discussion #15923) you would just need to change the image in the unraid config, no reason to find another app or make changes there — Reply to this email directly, view it on GitHub <#15923 (reply in thread)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGGTP5WKSE3T7HYLDYNPZAD2J7EZRAVCNFSM6AAAAABU5P7VGCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCNZZHAZTGMA> . You are receiving this because you authored the thread. <https://github.com/notifications/beacon/AGGTP5WXIU56HMYXYZ5NFA32J7EZRA5CNFSM6AAAAABU5P7VGCWGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAWQDTU.gif> Message ID: ***@***.*** ***@***.***> >

[NickM-27]

I would suggest removing

  input_tensor: nhwc

  input_pixel_format: bgr

  labelmap_path: /config/models/coco_labels.txt  # Correct path for label map

  path: /config/models/efficientdet_lite0.tflite  # Correct path for the Coral TPU model

as there is no reason to override these fields