[Detector Support]: Slow Coral Performance on Dell R720XD/Proxmox/UbuntuVM/Docker

[kranked]

Describe the problem you are having

Hi, received my Coral yesterday and unsure if I've set it up wrong or just going through a Dell R720XD USB2 --> proxmox --> ubuntu VM/docker to Frigate is causing slow performance. Detector inference is sitting around 125ms and Detector CPU Usage around 100%. I passed the Coral through to the VM and installed the google drivers after adding the repo. In my docker config I have:

devices:
- /dev/bus/usb:/dev/bus/usb # Passes the USB Coral, needs to be modified for other versions
-
And in my Frigate config I have:

detectors:
coral:
type: edgetpu
device: usb

That is all relating to detection apart from 'detect' role for each camera.
Have I missed anything?

Version

0.14.1-f4f3cfa

Frigate config file

mqtt:
  enabled: true
  host: 192.168.3.27
  user: redacted
  password: redacted

record:
  enabled: true
  retain:
    days: 30
    mode: all

go2rtc:
  rtsp:
    username: redacted
    password: redacted
  streams:
    Kitchen-Security:
      - rtsp://redacted:[email protected]/stream1
    Lounge-Security:
      - rtsp://redacted:[email protected]/stream1
    Yoyo-Room-Security:
      - rtsp://redacted:[email protected]/stream1

birdseye:
  enabled: false
  restream: true
  quality: 1
  height: 1080
  width: 1920
  mode: continuous

cameras:
  Kitchen-Security: # <------ Name the camera
    enabled: true
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/Kitchen-Security # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    onvif:
      host: 192.168.7.106
      port: 2020
      user: redacted
      password: redacted
    motion:
      mask: 0.313,0,0,0,0,0.086,0.316,0.088
  Lounge-Security:
    enabled: true
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/Lounge-Security # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    onvif:
      host: 192.168.7.107
      port: 2020
      user: redacted
      password: redacted
    motion:
      mask: 0,0.094,0.318,0.101,0.317,0,0,0
  Yoyo-Room-Security:
    enabled: true
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/Yoyo-Room-Security # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    onvif:
      host: 192.168.7.140
      port: 2020
      user: redacted
      password: redacted
    detect:
      enabled: false # <---- disable detection until you have a working camera feed
      width: 1280
      height: 720

    motion:
      mask: 0.002,0.001,0.317,0,0.317,0.09,0,0.095

detectors:
  coral:
    type: edgetpu
    device: usb

camera_groups:
  Birdseye:
    order: 1
    icon: LuAperture
    cameras: birdseye

version: 0.14

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "2gb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # Passes the USB Coral, needs to be modified for other versions
    volumes:
      - ./config:/config
      - /mnt/frigate-nvr-record/storage:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "8971:8971"
      - "8554:8554" # RTSP feeds
      - "8555:8555" # WebRTC connections for low latency live views.
      - "5000:5000" # Internal unathenticated access - use firewall rules on server.

Relevant Frigate log output

Coral is very slow, 123ms. Log states TPU is detected so I don't think there's a config problem.

Install method

Proxmox via Docker

Object Detector

Coral

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

This is a VM on a Dell R720XD hypervisor, it has 8 CPU's assigned and 32GB of RAM. I have the Coral attached to a USB slot and have passed through the USB slot to the Frigate VM, which is Ubuntu with Docker/Portainer installed. There is no GPU yet so no hardware accel for encoding so I purchased a Coral this week to speed up inference and it seems to be struggling, if indeed it's using it at all.
I installed the standard drivers for the Coral, not the overclocked one.

[hawkeye217]

Though Proxmox is not an officially supported installation method, many Proxmox users have had better luck installing Frigate as an LXC using the tteck script.

[roger-]

Noticing a similar issue with Proxmox. Did you ever find a solution?

[willdab345t]

Hi @roger- Yes I installed a PCI to USB3.0 card in the server for the Coral card, this elimated most of the latency. I also configured a few settings for the Frigate VM, can't remember what exactly now but all common optimisation suggestions you'll find online, wasn't much. Inference latency sits around 25MS now, maybe too high for some people but the performance is absolutely fine, no image delays etc, I am runny only 6 cameras though around 1440p. I doubt it can be brought down much more on the R720XD in a VM, just too many I/O bottlenecks as it's an ancient server at this point and I don't have a GPU installed for GPU offloading so the detector usage goes high quite often when there's a lot of action.
So 95% of the resolution was not enough bandwidth on USB 2.0 bus for the Coral card.