[HW Accel Support]: Am I actually using hardware accelleration? #14181

jftanner · 2024-10-06T17:37:59Z

jftanner
Oct 6, 2024

Describe the problem you are having

Hello!

I'm recently moved Frigate from a little micro-pc to an old gaming rig I've converted into a homelab. It's working quite well, and I think everything is set up properly, but there's one thing I'm still not entirely confident of: I can't tell if I'm actually using hardware acceleration or not. I see ffmpeg processes on the GPU, but the utilization is extremely low: 1-3%

nvidia-smi on the host OS:

Frigate itself reports 0.0% GPU usage, but I think that's a permission issue. It sees both GPUs (which is odd), but not their processes.
nvidia-smi in the container:

I'm not sure if everything is totally fine, and the cameras just aren't any meaningful load for the 980 Ti, or if it's not actually being used the way it should be. I'd appreciate any advice.

And thank you, as always, for such a fantastic piece of software. Frigate is vastly better than the Lorex NVR that came with the cameras.

Version

0.14.1-f4f3cfa

Frigate config file

version: 0.14

# Redacted for security: auth, proxy, mqtt

ffmpeg:
  hwaccel_args: preset-nvidia-h264
  input_args: preset-rtsp-restream
  output_args:
    record: preset-record-generic-audio-copy

detect:
  width: 1920
  height: 1080
  fps: 5

detectors:
  coral:
    type: edgetpu
    device: pci:0

model:
  path: plus://REDACTED

# Removed for brevity: objects, review

record:
  enabled: true
  events:
    pre_capture: 10
    post_capture: 15
    retain:
      mode: active_objects

snapshots:
  enabled: true
  timestamp: true
  bounding_box: true
  crop: false

go2rtc:
  streams:
    driveway_cam:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=1&subtype=0
      - ffmpeg:driveway_cam#audio=opus
    driveway_cam_sub:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=1&subtype=2
    front_cam:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=2&subtype=0
      - ffmpeg:front_cam#audio=opus
    front_cam_sub:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=2&subtype=2
    porch_cam:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=3&subtype=0
      - ffmpeg:porch_cam#audio=opus
    porch_cam_sub:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=3&subtype=2
    backyard_cam:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=4&subtype=0
      - ffmpeg:backyard_cam#audio=opus
    backyard_cam_sub:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=4&subtype=2
    garden_cam:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=5&subtype=0
      - ffmpeg:garden_cam#audio=opus
    garden_cam_sub:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@REDACTED/cam/realmonitor?channel=5&subtype=2
  webrtc:
    candidates:
      - REDACTED:8555
      - stun:8555

cameras:
  driveway_cam:
    ui:
      order: 1
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/driveway_cam_sub
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/driveway_cam
          input_args: preset-rtsp-restream
          roles:
            - record
    record:
      retain:
        days: 0.25
        mode: motion
    # Removed for brevity: snapshots, review, motion, zones

  front_cam:
    ui:
      order: 2
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_cam_sub
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/front_cam
          roles:
            - record
    # Removed for brevity: snapshots, review, motion, zones

  porch_cam:
    ui:
      order: 3
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/porch_cam_sub
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/porch_cam
          roles:
            - record
    # Removed for brevity: objects snapshots, review, motion, zones

  backyard_cam:
    ui:
      order: 4
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/backyard_cam_sub
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/backyard_cam
          roles:
            - record
    # Removed for brevity: objects, review, motion, zones

  garden_cam:
    ui:
      order: 5
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garden_cam_sub
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/garden_cam
          roles:
            - record
    # Removed for brevity: objects, review, motion, zones

docker-compose file or Docker CLI command

# This is the deployment manifest I'm using.
# It's probably the only relevant file, but I can provide more information as needed.
apiVersion: apps/v1
kind: Deployment
metadata:
  name:  frigate
  namespace: frigate
  labels:
    app:  frigate
spec:
  selector:
    matchLabels:
      app: frigate
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app:  frigate
    spec:
      runtimeClassName: nvidia
      containers:
      - name:  frigate
        image:  ghcr.io/blakeblackshear/frigate:0.14.1
        securityContext:
          privileged: true
          # Pending https://github.com/blakeblackshear/frigate/issues/3108
          # runAsUser: 112
          # runAsGroup: 113
        envFrom:
        - secretRef:
            name: frigate-secrets
        ports:
        - containerPort: 5000
          name: http-open
        - containerPort: 8971
          name: http-auth
        - containerPort: 8554
          name: rtsp
        - containerPort: 8555
          protocol: TCP
          name: webrtc-tcp
        - containerPort: 8555
          protocol: UDP
          name: webrtc-udp
        resources:
          requests:
            cpu: '2'
            memory: '6Gi'
            nvidia.com/gpu: '1'
          limits:
            cpu: '4'
            memory: '12Gi'
            nvidia.com/gpu: '1'
        livenessProbe:
          initialDelaySeconds: 30
          failureThreshold: 5
          timeoutSeconds: 10
          httpGet:
            path: /
            port: http-open
            scheme: HTTP
        readinessProbe:
          initialDelaySeconds: 15
          timeoutSeconds: 5
          httpGet:
            path: /
            port: http-open
            scheme: HTTP
        volumeMounts:
        # Config directory (config.yaml + database), written to /store
        - mountPath: /config
          name: config
        # Exports directory, written to /store
        - mountPath: /media/frigate/exports
          name: exports
        # Media directories, in a pvc
        - mountPath: /media/frigate/clips # TODO Remove this after rename
          name: media
          subPath: snapshots
        - mountPath: /media/frigate/snapshots
          name: media
          subPath: snapshots
        - mountPath: /media/frigate/recordings
          name: media
          subPath: recordings
        # Coral TPU
        - mountPath: /dev/apex_0
          name: coral
        # Local time zone
        - mountPath: /etc/localtime
          name: localtime
          readOnly: true
        # Working space
        - mountPath: /tmp/cache
          name: tmp
        - mountPath: /dev/shm
          name: shm
      volumes:
      - name: config
        hostPath:
          path: /store/frigate/config
          type: Directory
      - name: exports
        hostPath:
          path: /store/frigate/exports
          type: Directory
      - name: media
        persistentVolumeClaim:
          claimName: frigate-media-pvc
      - name: coral
        hostPath:
          path: /dev/apex_0
          type: CharDevice
      - name: localtime
        hostPath:
          path: /etc/localtime
          type: File
      - name: tmp
        emptyDir:
          medium: Memory
          sizeLimit: 2Gi
      - name: shm
        emptyDir:
          medium: Memory
          sizeLimit: 512Mi

Relevant Frigate log output

2024-10-06 13:24:21.545281121  [INFO] Preparing Frigate...
2024-10-06 13:24:21.546385870  [INFO] Starting NGINX...
[WARN] /sys/fs/cgroup/cpu.max not found. Falling back to /proc/cpuinfo.
2024-10-06 13:24:21.548185535  [INFO] Removing stale config from last run...
2024-10-06 13:24:21.548923704  [INFO] Preparing new go2rtc config...
2024-10-06 13:24:21.551568991  [INFO] No TLS certificate found. Generating a self signed certificate...
2024-10-06 13:24:21.554297168  [INFO] Starting Frigate...
2024-10-06 13:24:21.864230971  [INFO] Not injecting WebRTC candidates into go2rtc config as it has been set manually
2024-10-06 13:24:21.933024095  [INFO] Starting go2rtc...
s6-rc: info: service nginx successfully started
s6-rc: info: service certsync: starting
s6-rc: info: service certsync successfully started
s6-rc: info: service legacy-services: starting
2024-10-06 13:24:22.008324477  [INFO] Starting certsync...
s6-rc: info: service legacy-services successfully started
2024-10-06 13:24:22.052647430  13:24:22.052 INF go2rtc platform=linux/amd64 revision=b2399f3 version=1.9.2
2024-10-06 13:24:22.052663315  13:24:22.052 INF config path=/dev/shm/go2rtc.yaml
2024-10-06 13:24:22.053072191  13:24:22.053 INF [api] listen addr=:1984
2024-10-06 13:24:22.053123481  13:24:22.053 INF [rtsp] listen addr=:8554
2024-10-06 13:24:22.053437245  13:24:22.053 INF [webrtc] listen addr=:8555
2024-10-06 13:24:22.078767613  127.0.0.1 - - [06/Oct/2024:13:24:22 -0400] "" 400 0 "-" "-" "-"
2024-10-06 13:24:22.941015953  [2024-10-06 13:24:22] frigate.app                    INFO    : Starting Frigate (0.14.1-f4f3cfa)
2024-10-06 13:24:22.941100548  [2024-10-06 13:24:22] frigate.util.config            INFO    : Checking if frigate config needs migration...
2024-10-06 13:24:22.996571829  [2024-10-06 13:24:22] frigate.util.config            INFO    : frigate config does not need migration...
2024-10-06 13:24:23.056480162  [2024-10-06 13:24:23] peewee_migrate.logs            INFO    : Starting migrations
2024-10-06 13:24:23.057064145  [2024-10-06 13:24:23] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-10-06 13:24:23.077109140  [2024-10-06 13:24:23] frigate.app                    INFO    : Recording process started: 356
2024-10-06 13:24:23.078025903  [2024-10-06 13:24:23] frigate.app                    INFO    : Recording process started: 365
2024-10-06 13:24:23.080154492  [2024-10-06 13:24:23] frigate.app                    INFO    : go2rtc process pid: 98
2024-10-06 13:24:23.104040191  [2024-10-06 13:24:23] detector.coral                 INFO    : Starting detection process: 385
2024-10-06 13:24:23.119291678  [2024-10-06 13:24:23] frigate.app                    INFO    : Output process started: 388
2024-10-06 13:24:23.119462052  [2024-10-06 13:24:23] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci:0
2024-10-06 13:24:23.119599200  [2024-10-06 13:24:23] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2024-10-06 13:24:23.157634785  [2024-10-06 13:24:23] frigate.app                    INFO    : Camera processor started for driveway_cam: 424
2024-10-06 13:24:23.165305548  [2024-10-06 13:24:23] frigate.app                    INFO    : Camera processor started for front_cam: 427
2024-10-06 13:24:23.173300918  [2024-10-06 13:24:23] frigate.app                    INFO    : Camera processor started for porch_cam: 432
2024-10-06 13:24:23.183208634  [2024-10-06 13:24:23] frigate.app                    INFO    : Camera processor started for backyard_cam: 441
2024-10-06 13:24:23.191927981  [2024-10-06 13:24:23] frigate.app                    INFO    : Camera processor started for garden_cam: 449
2024-10-06 13:24:23.209251700  [2024-10-06 13:24:23] frigate.app                    INFO    : Capture process started for driveway_cam: 457
2024-10-06 13:24:23.209389853  [2024-10-06 13:24:23] frigate.app                    INFO    : Capture process started for front_cam: 467
2024-10-06 13:24:23.262727320  [2024-10-06 13:24:23] frigate.app                    INFO    : Capture process started for porch_cam: 472
2024-10-06 13:24:23.262732417  [2024-10-06 13:24:23] frigate.app                    INFO    : Capture process started for backyard_cam: 477
2024-10-06 13:24:23.262737312  [2024-10-06 13:24:23] frigate.app                    INFO    : Capture process started for garden_cam: 493

Relevant go2rtc log output

2024-10-06 13:24:21.548923704  [INFO] Preparing new go2rtc config...
2024-10-06 13:24:21.864230971  [INFO] Not injecting WebRTC candidates into go2rtc config as it has been set manually
2024-10-06 13:24:21.933024095  [INFO] Starting go2rtc...
2024-10-06 13:24:22.052647430  13:24:22.052 INF go2rtc platform=linux/amd64 revision=b2399f3 version=1.9.2
2024-10-06 13:24:22.052663315  13:24:22.052 INF config path=/dev/shm/go2rtc.yaml
2024-10-06 13:24:22.053072191  13:24:22.053 INF [api] listen addr=:1984
2024-10-06 13:24:22.053123481  13:24:22.053 INF [rtsp] listen addr=:8554
2024-10-06 13:24:22.053437245  13:24:22.053 INF [webrtc] listen addr=:8555
2024-10-06 13:24:31.545220314  [INFO] Starting go2rtc healthcheck service...

FFprobe output from your camera

ffprobe version n5.1-2-g915ef932a3-20220731 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 12.1.0 (crosstool-NG 1.25.0.55_3defb7b)
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- --arch=x86_64 --target-os=linux --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libass --enable-libbluray --enable-libjxl --enable-libmp3lame --enable-libopus --enable-mbedtls --enable-librist --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --enable-libplacebo --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lgomp' --extra-version=20220731
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100

Install method

Kubernetes manifests

Object Detector

Coral

Network connection

Wireless

Camera make and model

Lorex N843A8 NVR with 4x E892AB + 1x E893AB

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

Host stack:

Host OS is Ubuntu Server 24.04.
A PCIe Coral TPU is installed in the host OS.
Nvidia drivers are installed in the host OS, though Ubuntu itself is using onboard graphics instead.
Running Kubernetes via k3s.
Nvidia support is added to k3s via the Nvidia operator.
Frigate is installed in Kubernetes via custom manifests.
Frigate is allocated a single GPU and also mounts the Coral TPU.

(I opted for custom manifests rather than the semi-official Helm chart, because it was just easier to configure everything the way I wanted it. Especially persistent storage for media and the config.)

Answered by NickM-27

Oct 6, 2024

Yes, decoding a camera stream is light work for a discrete GPU.

View full answer

NickM-27 · 2024-10-06T17:39:53Z

NickM-27
Oct 6, 2024
Collaborator Sponsor

It shows the ffmpeg processes so everything is working correctly. If I had to guess the stats are just showing stats for the other GPU.

3 replies

jftanner Oct 6, 2024
Author

(Copied from another comment, because I didn't realize discussions were threaded.)

Wow! What an incredibly prompt reply!
I suspect you're right about the in-Frigate stats showing 0%. That's certainly as good a reason as any. But it's still <3% usage on the GPU it's actually using.

I guess I just don't really understand what the GPU is used for, or how much usage is expected. Maybe it just really doesn't need to work very hard? (Which I suppose would make sense, since it did work on the little micro-PC.) If that's the case, are there any settings I can increase without overloading the TPU or harming detection performance?

These are the camera's stream settings, if it helps:

NickM-27 Oct 6, 2024
Collaborator Sponsor

Yes, decoding a camera stream is light work for a discrete GPU.

Answer selected by jftanner

jftanner Oct 6, 2024
Author

Makes sense.
Much ado about nothing then. :)

Thanks for your help!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[HW Accel Support]: Am I actually using hardware accelleration? #14181

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

[HW Accel Support]: Am I actually using hardware accelleration? #14181

Uh oh!

Uh oh!

jftanner Oct 6, 2024

Describe the problem you are having

Version

Frigate config file

docker-compose file or Docker CLI command

Relevant Frigate log output

Relevant go2rtc log output

FFprobe output from your camera

Install method

Object Detector

Network connection

Camera make and model

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

Replies: 1 comment · 3 replies

Uh oh!

Uh oh!

NickM-27 Oct 6, 2024 Collaborator Sponsor

Uh oh!

jftanner Oct 6, 2024 Author

Uh oh!

NickM-27 Oct 6, 2024 Collaborator Sponsor

Uh oh!

jftanner Oct 6, 2024 Author

jftanner
Oct 6, 2024

Replies: 1 comment 3 replies

NickM-27
Oct 6, 2024
Collaborator Sponsor

jftanner Oct 6, 2024
Author

NickM-27 Oct 6, 2024
Collaborator Sponsor

jftanner Oct 6, 2024
Author