[HW Accel Support]: Intel i7-12700T vGPU Passthrough to Ubuntu - not working in Frigate?

[captainkev76]

Describe the problem you are having

Hi all,

I've just done a fresh install of Proxmox on new hardware (i7-12700T) . I have an Ubuntu 22.04 VM running on Proxmox, and have followed [Derek Seaman's] (https://www.derekseaman.com/2023/06/proxmox-ve-8-windows-11-vgpu-vt-d-passthrough-with-intel-alder-lake-legacy.html) guide for passing through virtual GPUs to VMs.

That seems to be working on the Proxmox side, and in the Ubuntu VM I see:

00:01.0 VGA compatible controller: Device 1234:1111 (rev 02)
06:1b.0 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)

But in Frigate (which is running in docker on Ubuntu), I see lots of errors such as:

No VA display found for device /dev/dri/renderD128. 
Failed to set value '/dev/dri/renderD128' for option 'qsv_device': Invalid argument 

which suggest that there's a problem with the vGPU passthrough.

I've tried changing "hwaccel_args: preset-intel-qsv-h264" to "hwaccel_args: preset-vaapi", but to no avail. I now get these errors instead:

2024-03-03 20:46:02.466318133  [2024-03-03 20:46:02] frigate.video                  ERROR   : garage: Unable to read frames from ffmpeg process.
2024-03-03 20:46:02.466385945  [2024-03-03 20:46:02] frigate.video                  ERROR   : garage: ffmpeg process is not running. exiting capture thread...
2024-03-03 20:46:02.571933179  [2024-03-03 20:46:02] frigate.util.services          ERROR   : Unable to poll intel GPU stats: No device filter specified and no discrete/integrated i915 devices found

Wondering if anyone has come across this before or might know of a solution?

Kevin

Version

0.13.2-6476f8a

Frigate config file

ffmpeg:
  hwaccel_args: preset-vaapi
mqtt:
  enabled: True
  user: "redacted"
  password: "redacted"
  host: 192.168.1.94
  topic_prefix: frigate
  client_id: frigate

detectors:
  coral:
    type: edgetpu
    device: pci
cameras:
  driveway:
    ffmpeg:
      inputs:
        - path: rtsp://[redacted]@192.168.50.50:554/h264Preview_01_sub # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://[redacted]@192.168.50.50:554/h264Preview_01_main
          roles:
            - record
    detect:
      enabled: True # <---- disable detection until you have a working camera feed
      width: 640 # <---- update for your camera's resolution
      height: 480 # <---- update for your camera's resolution
    motion:
      mask:
        - 0,0,127,0,205,0,222,40,175,259,152,480,0,480
    record:
      enabled: True
      retain:
        days: 30
        mode: motion
      events:
        retain:
          default: 90
          mode: active_objects
          objects:
            car: 30
            person: 90
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://[redacted]@192.168.50.53:554/h264Preview_01_sub # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://[redacted]@192.168.50.53:554/h264Preview_01_main
          roles:
            - record
    detect:
      enabled: True # <---- disable detection until you have a working camera feed
      width: 640 # <---- update for your camera's resolution
      height: 480 # <---- update for your camera's resolution
    record:
      enabled: True
      retain:
        days: 30
        mode: motion
      events:
        retain:
          default: 90
          mode: active_objects
          objects:
            car: 30
            person: 90
  back_garden:
    ffmpeg:
      inputs:
        - path: rtsp://[redacted]@192.168.50.52:554/h264Preview_01_sub # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://[redacted]@192.168.50.52:554/h264Preview_01_main
          roles:
            - record
    detect:
      enabled: True # <---- disable detection until you have a working camera feed
      width: 640 # <---- update for your camera's resolution
      height: 480 # <---- update for your camera's resolution
    record:
      enabled: True
      retain:
        days: 30
        mode: motion
      events:
        retain:
          default: 30
          mode: active_objects
          objects:
            car: 30
            person: 90
  side_path:
    ffmpeg:
      inputs:
        - path: rtsp://[redacted]@192.168.50.51:554/h264Preview_01_sub # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://[redacted]@192.168.50.51:554/h264Preview_01_main
          roles:
            - record
    motion:
      mask:
        - 0,0,127,0,205,0,222,40,175,259,152,480,0,480
    detect:
      enabled: True # <---- disable detection until you have a working camera feed
      width: 640 # <---- update for your camera's resolution
      height: 480 # <---- update for your camera's resolution
    record:
      enabled: True
      retain:
        days: 90
        mode: motion
      events:
        retain:
          default: 30
          mode: active_objects
          objects:
            car: 30
            person: 90
  garage:
    ffmpeg:
      inputs:
        - path: rtsp://[redacted]@192.168.50.55/live0 # <----- The stream you want to use for detection
          roles:
            - detect
        - path: rtsp://[redacted]@192.168.50.55/live0
          roles:
            - record
    detect:
      enabled: True # <---- disable detection until you have a working camera feed
      width: 1920 # <---- update for your camera's resolution
      height: 1080 # <---- update for your camera's resolution
    record:
      enabled: True
      retain:
        days: 90
        mode: motion
      events:
        retain:
          default: 30
          mode: active_objects
          objects:
            person: 90

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "1024M" 
    devices:
      - /dev/apex_0:/dev/apex_0 
      - /dev/dri/renderD128 
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /mnt/docker/docker/frigate:/config
      - /mnt/cctv/frigate:/media/frigate
      - type: tmpfs 
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "redacted"

Relevant log output

2024-03-03 21:13:20.547922228  [2024-03-03 21:13:20] frigate.video                  ERROR   : side_path: Unable to read frames from ffmpeg process.
2024-03-03 21:13:20.548567663  [2024-03-03 21:13:20] frigate.video                  ERROR   : side_path: ffmpeg process is not running. exiting capture thread...
2024-03-03 21:13:20.549066846  [2024-03-03 21:13:20] frigate.video                  ERROR   : front_door: Unable to read frames from ffmpeg process.
2024-03-03 21:13:20.549229000  [2024-03-03 21:13:20] frigate.video                  ERROR   : front_door: ffmpeg process is not running. exiting capture thread...
2024-03-03 21:13:20.559299711  [2024-03-03 21:13:20] frigate.video                  ERROR   : garage: Unable to read frames from ffmpeg process.
2024-03-03 21:13:20.559475991  [2024-03-03 21:13:20] frigate.video                  ERROR   : garage: ffmpeg process is not running. exiting capture thread...
2024-03-03 21:13:20.565256171  [2024-03-03 21:13:20] frigate.video                  ERROR   : driveway: Unable to read frames from ffmpeg process.
2024-03-03 21:13:20.565408182  [2024-03-03 21:13:20] frigate.video                  ERROR   : driveway: ffmpeg process is not running. exiting capture thread...
2024-03-03 21:13:20.572618427  [2024-03-03 21:13:20] frigate.video                  ERROR   : back_garden: Unable to read frames from ffmpeg process.
2024-03-03 21:13:20.573119799  [2024-03-03 21:13:20] frigate.video                  ERROR   : back_garden: ffmpeg process is not running. exiting capture thread...
2024-03-03 21:13:20.604558470  [2024-03-03 21:13:20] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as pci
2024-03-03 21:13:20.607580902  [2024-03-03 21:13:20] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2024-03-03 21:13:22.568395952  [2024-03-03 21:13:22] frigate.util.services          ERROR   : Unable to poll intel GPU stats: No device filter specified and no discrete/integrated i915 devices found
2024-03-03 21:13:22.568398070

FFprobe output from your camera

ffprobe version 4.4.2-0ubuntu0.22.04.1+esm3 Copyright (c) 2007-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1+esm3 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rtsp, from 'rtsp://[redacted]192.168.50.50:554/h264Preview_01_sub':
  Metadata:
    title           : Session streamed by "preview"
    comment         : h264Preview_01_sub
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 640x480, 29.92 tbr, 90k tbn, 180k tbc
  Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp

Operating system

Proxmox

Install method

Docker Compose

Network connection

Wired

Camera make and model

Reolink - various

Any other information that may be helpful

No response

[captainkev76]

Anyone got any thoughts on this one? I'd expect that the i7-12700T CPU must be used quite widely. Is there an alternative way to pass through the iGPU without virtualising it?