[HW Accel Support]: Proxmox Ryzen Frigate 14 crashes if I try to enable HW acceleration on top of Coral USB TPU

[xabix]

Describe the problem you are having

Hello,

My system seems to start and then crashes and restart in a loop. It was working well with Coral only and privileged rights in docker but as I installed a new CPU with builtin GPU AMD Ryzen 7 9700X and wanted to take benefit of it and also bc sometimes with proxmox the coral in USB get lost so having also CPU detectors seem to provide better resiliency.

Should i be trying with renderD128 or card1?

Below my proxmox conf:

Version

0.14.1-f4f3cfa

Frigate config file

mqtt:
  enabled: true
  topic_prefix: frigate
  client_id: frigate

ffmpeg:
  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 #same with card1 deviceEnable Hardware Acceleration
  global_args: -hide_banner -loglevel error -threads 4
  output_args:
    detect: -threads 4 -f rawvideo -pix_fmt yuv420p
    record: preset-record-generic
  retry_interval: 10

logger:
  default: info

  header_map:
    user: x-forwarded-user

record:
  enabled: true
  retain:
    days: 30
    mode: active_objects
  events:
    pre_capture: 5
    post_capture: 5
    retain:
      default: 30
      mode: active_objects

snapshots:
  enabled: true
  clean_copy: true
  timestamp: true
  bounding_box: true
  crop: false
  retain:
    default: 15

go2rtc:
  # log:
  #   exec: trace
  streams:
    rtsp_GarageDehors: rtsp://@192.168.20.6:554/ch1/main/av_stream
    rtsp_GarageDehors_sub: rtsp://[email protected]:554/ISAPI/Streaming/channels/102/httpPreview
   etc....

cameras:
  GarageDehors:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/rtsp_GarageDehors
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/rtsp_GarageDehors_sub
          roles:
            - detect
    detect:
      enabled: true
    live:
      stream_name: rtsp_GarageDehors
    motion:
      mask:
        - 1920,0,0,0,0,402,802,188,1920,572
    objects:
      track:
        - person
        - bicycle
        - motorcycle
      filters:
        bicycle:
          mask:
        motorcycle:
          mask:
        person:
          mask: 0.512,0.109,0.54,0.11,0.538,0.211,0.513,0.213
    ui:
      order: 0
etc....

birdseye:
  enabled: true
  restream: true
  quality: 4
  mode: continuous

detect:
  width: 1920 
  height: 1080 
  fps: 5
  enabled: true
  min_initialized: 2
  max_disappeared: 25
    max_frames:
      default: 3000
      objects:
        person: 1000

detectors:
  coral:
    type: edgetpu
    device: usb
  cpu:
    type: cpu
    num_threads: 4

model:
  width: 320
  height: 320
  labelmap:
    0: person
    1: bicycle
    2: car
    3: motorcycle
    4: airplane
    5: bus
    6: train
    7: car
    8: boat
    9: traffic light
    10: fire hydrant
    11: street sign
    12: stop sign
    13: parking meter
    14: bench
    15: bird
    16: cat
    17: dog
    18: horse
    19: sheep
    20: cow
    21: elephant
    22: bear
    23: zebra
    24: giraffe
    25: hat
    26: backpack
    27: umbrella
    28: shoe
    29: eye glasses
    30: handbag
    31: tie
    32: suitcase
    33: frisbee
    34: skis
    35: snowboard
    36: sports ball
    37: kite
    38: baseball bat
    39: baseball glove
    40: skateboard
    41: surfboard
    42: tennis racket
    43: bottle
    44: plate
    45: wine glass
    46: cup
    47: fork
    48: knife
    49: spoon
    50: bowl
    51: banana
    52: apple
    53: sandwich
    54: orange
    55: broccoli
    56: carrot
    57: hot dog
    58: pizza
    59: donut
    60: cake
    61: chair
    62: couch
    63: potted plant
    64: bed
    65: mirror
    66: dining table
    67: window
    68: desk
    69: toilet
    70: door
    71: tv
    72: laptop
    73: mouse
    74: remote
    75: keyboard
    76: cell phone
    77: microwave
    78: oven
    79: toaster
    80: sink
    81: refrigerator
    82: blender
    83: book
    84: clock
    85: vase
    86: scissors
    87: teddy bear
    88: hair drier
    89: toothbrush
    90: hair brush

version: 0.14
camera_groups:
  Birdseye:
    order: 1
    icon: LuBird
    cameras: birdseye

docker-compose file or Docker CLI command

frigate:
    container_name: frigate
    image: ghcr.io/blakeblackshear/frigate:stable
    hostname: frigate
    restart: unless-stopped
    shm_size: "256mb"
    cap_add:
      - CAP_PERFMON
      - SYS_ADMIN 
    volumes:
      - $DOCKERDIR/appdata/frigate/config:/config
      - /opt/Crucial_SSD_256Go:/media/frigate
      - /dev/bus/usb:/dev/bus/usb
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    devices:
      - /dev/dri/card1:/dev/dri/card1
      - /dev/dri/renderD128:/dev/dri/renderD128
    networks:
      - my_bridge
    ports:
      - "1935:1935" # RTMP feeds
      - "5000:5000" # Port used by the Web UI
      - "5001:5001" # Auth
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
      - "8971:8971" # Authenticated UI and API. Reverse proxies should use this port
    environment:
      FRIGATE_RTSP_PASSWORD: xxx
      TZ: $TZ
      PLUS_API_KEY: xxx
      LIBVA_DRIVER_NAME: radeonsi

Relevant Frigate log output

2024-12-12 09:34:33.294611194  [2024-12-12 10:34:33] watchdog.Jardin                ERROR   : Ffmpeg process crashed unexpectedly for Jardin.
2024-12-12 09:34:33.294612176  [2024-12-12 10:34:33] watchdog.Jardin                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-12 09:34:33.294612837  [2024-12-12 10:34:33] ffmpeg.Jardin.detect           ERROR   : [AVHWDeviceContext @ 0x5747e216cb40] libva: /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so init failed
2024-12-12 09:34:33.294613378  [2024-12-12 10:34:33] ffmpeg.Jardin.detect           ERROR   : [AVHWDeviceContext @ 0x5747e216cb40] Failed to initialise VAAPI connection: 2 (resource allocation failed).
2024-12-12 09:34:33.294613849  [2024-12-12 10:34:33] ffmpeg.Jardin.detect           ERROR   : Device creation failed: -5.
2024-12-12 09:34:33.294614380  [2024-12-12 10:34:33] ffmpeg.Jardin.detect           ERROR   : [h264 @ 0x5747e200c4c0] No device available for decoder: device type vaapi needed for codec h264.
2024-12-12 09:34:33.294614871  [2024-12-12 10:34:33] ffmpeg.Jardin.detect           ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
2024-12-12 09:34:33.294760605  [2024-12-12 10:34:33] watchdog.Garage                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-12 09:34:33.294761326  [2024-12-12 10:34:33] ffmpeg.Garage.detect           ERROR   : [AVHWDeviceContext @ 0x5db259921a80] libva: /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so init failed
2024-12-12 09:34:33.294761867  [2024-12-12 10:34:33] ffmpeg.Garage.detect           ERROR   : [AVHWDeviceContext @ 0x5db259921a80] Failed to initialise VAAPI connection: 2 (resource allocation failed).
2024-12-12 09:34:33.294762258  [2024-12-12 10:34:33] ffmpeg.Garage.detect           ERROR   : Device creation failed: -5.
2024-12-12 09:34:33.294762779  [2024-12-12 10:34:33] ffmpeg.Garage.detect           ERROR   : [h264 @ 0x5db2599348c0] No device available for decoder: device type vaapi needed for codec h264.
2024-12-12 09:34:33.294763260  [2024-12-12 10:34:33] ffmpeg.Garage.detect           ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
2024-12-12 09:34:33.296744979  [2024-12-12 10:34:33] watchdog.Bureau                ERROR   : Ffmpeg process crashed unexpectedly for Bureau.
2024-12-12 09:34:33.296746763  [2024-12-12 10:34:33] watchdog.Bureau                ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-12 09:34:33.296747604  [2024-12-12 10:34:33] ffmpeg.Bureau.detect           ERROR   : [AVHWDeviceContext @ 0x6083d10f0c00] libva: /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so init failed
2024-12-12 09:34:33.296748165  [2024-12-12 10:34:33] ffmpeg.Bureau.detect           ERROR   : [AVHWDeviceContext @ 0x6083d10f0c00] Failed to initialise VAAPI connection: 2 (resource allocation failed).
2024-12-12 09:34:33.296748616  [2024-12-12 10:34:33] ffmpeg.Bureau.detect           ERROR   : Device creation failed: -5.
2024-12-12 09:34:33.296749137  [2024-12-12 10:34:33] ffmpeg.Bureau.detect           ERROR   : [h264 @ 0x6083d0fdae40] No device available for decoder: device type vaapi needed for codec h264.
2024-12-12 09:34:33.296749598  [2024-12-12 10:34:33] ffmpeg.Bureau.detect           ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
2024-12-12 09:34:33.299823427  [2024-12-12 10:34:33] watchdog.Salon                 ERROR   : Ffmpeg process crashed unexpectedly for Salon.
2024-12-12 09:34:33.299825360  [2024-12-12 10:34:33] watchdog.Salon                 ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-12 09:34:33.299826192  [2024-12-12 10:34:33] ffmpeg.Salon.detect            ERROR   : [AVHWDeviceContext @ 0x5979c759ac80] libva: /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so init failed
2024-12-12 09:34:33.299826763  [2024-12-12 10:34:33] ffmpeg.Salon.detect            ERROR   : [AVHWDeviceContext @ 0x5979c759ac80] Failed to initialise VAAPI connection: 2 (resource allocation failed).
2024-12-12 09:34:33.299827174  [2024-12-12 10:34:33] ffmpeg.Salon.detect            ERROR   : Device creation failed: -5.
2024-12-12 09:34:33.299827725  [2024-12-12 10:34:33] ffmpeg.Salon.detect            ERROR   : [h264 @ 0x5979c7445d40] No device available for decoder: device type vaapi needed for codec h264.
2024-12-12 09:34:33.299828186  [2024-12-12 10:34:33] ffmpeg.Salon.detect            ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
2024-12-12 09:34:33.301400788  [2024-12-12 10:34:33] watchdog.RAB                   ERROR   : Ffmpeg process crashed unexpectedly for RAB.
2024-12-12 09:34:33.301449688  [2024-12-12 10:34:33] watchdog.RAB                   ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-12-12 09:34:33.301495625  [2024-12-12 10:34:33] ffmpeg.RAB.detect              ERROR   : [AVHWDeviceContext @ 0x58ce5c724000] libva: /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so init failed
2024-12-12 09:34:33.301527114  [2024-12-12 10:34:33] ffmpeg.RAB.detect              ERROR   : [AVHWDeviceContext @ 0x58ce5c724000] Failed to initialise VAAPI connection: 2 (resource allocation failed).
2024-12-12 09:34:33.301558292  [2024-12-12 10:34:33] ffmpeg.RAB.detect              ERROR   : Device creation failed: -5.
2024-12-12 09:34:33.301586085  [2024-12-12 10:34:33] ffmpeg.RAB.detect              ERROR   : [h264 @ 0x58ce5c736d40] No device available for decoder: device type vaapi needed for codec h264.
2024-12-12 09:34:33.301612354  [2024-12-12 10:34:33] ffmpeg.RAB.detect              ERROR   : Device setup failed for decoder on input stream #0:0 : Input/output error
2024-12-12 09:34:33.357781229  [2024-12-12 10:34:33] frigate.watchdog               INFO    : Detection appears to have stopped. Exiting Frigate...
s6-rc: info: service legacy-services: stopping
s6-rc: info: service legacy-services successfully stopped
s6-rc: info: service go2rtc-healthcheck: stopping
s6-rc: info: service certsync: stopping
2024-12-12 09:34:33.361302507  [INFO] The go2rtc-healthcheck service exited with code 256 (by signal 15)
s6-rc: info: service go2rtc-healthcheck successfully stopped
2024-12-12 09:34:33.361656943  [INFO] Service CERTSYNC exited with code 256 (by signal 15)
s6-rc: info: service certsync successfully stopped
s6-rc: info: service nginx: stopping
s6-rc: info: service certsync-log: stopping
s6-rc: info: service certsync-log successfully stopped
2024-12-12 09:34:33.389681818  [INFO] Service NGINX exited with code 0 (by signal 0)
s6-rc: info: service nginx successfully stopped
s6-rc: info: service nginx-log: stopping
s6-rc: info: service frigate: stopping
2024-12-12 09:34:33.390664663  [2024-12-12 10:34:33] frigate.app                    INFO    : Flask has exited...
s6-rc: info: service nginx-log successfully stopped
2024-12-12 09:34:33.390704768  [2024-12-12 10:34:33] frigate.app                    INFO    : Stopping...
2024-12-12 09:34:33.391293773  [2024-12-12 10:34:33] frigate.stats.emitter          INFO    : Exiting stats emitter...
2024-12-12 09:34:33.391316636  [2024-12-12 10:34:33] frigate.ptz.autotrack          INFO    : Exiting autotracker...
2024-12-12 09:34:33.391363163  [2024-12-12 10:34:33] frigate.events.cleanup         INFO    : Exiting event cleanup...
2024-12-12 09:34:33.391606952  [2024-12-12 10:34:33] frigate.record.cleanup         INFO    : Exiting recording cleanup...
2024-12-12 09:34:33.391640223  [2024-12-12 10:34:33] frigate.watchdog               INFO    : Exiting watchdog...
2024-12-12 09:34:33.391653388  [2024-12-12 10:34:33] frigate.storage                INFO    : Exiting storage maintainer...
2024-12-12 09:34:33.391885153  [2024-12-12 10:34:33] frigate.app                    INFO    : Waiting for capture process for GarageDehors to stop
2024-12-12 09:34:33.392810941  [2024-12-12 10:34:33] watchdog.GarageDehors          INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.392868789  [2024-12-12 10:34:33] watchdog.GarageDehors          INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.393072010  [2024-12-12 10:34:33] watchdog.GarageDehors          INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.393100323  [2024-12-12 10:34:33] watchdog.GarageDehors          INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.412086025  [2024-12-12 10:34:33] frigate.app                    INFO    : Waiting for capture process for Cote to stop
2024-12-12 09:34:33.412314624  [2024-12-12 10:34:33] watchdog.Cote                  INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.412315766  [2024-12-12 10:34:33] watchdog.Cote                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.413015689  [2024-12-12 10:34:33] watchdog.Cote                  INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.413051066  [2024-12-12 10:34:33] watchdog.Cote                  INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.479443729  [2024-12-12 10:34:33] frigate.app                    INFO    : Waiting for capture process for Piscine to stop
2024-12-12 09:34:33.479834832  [2024-12-12 10:34:33] watchdog.Piscine               INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.479836726  [2024-12-12 10:34:33] watchdog.Piscine               INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.480613183  [2024-12-12 10:34:33] watchdog.Piscine               INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.480652717  [2024-12-12 10:34:33] watchdog.Piscine               INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.490515428  [2024-12-12 10:34:33] frigate.app                    INFO    : Waiting for capture process for Bureau to stop
2024-12-12 09:34:33.491259885  [2024-12-12 10:34:33] watchdog.Bureau                INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.491261077  [2024-12-12 10:34:33] watchdog.Bureau                INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.491551091  [2024-12-12 10:34:33] watchdog.Bureau                INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.491627554  [2024-12-12 10:34:33] watchdog.Bureau                INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.558437169  [2024-12-12 10:34:33] frigate.app                    INFO    : Waiting for capture process for Garage to stop
2024-12-12 09:34:33.559370892  [2024-12-12 10:34:33] watchdog.Garage                INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.559372775  [2024-12-12 10:34:33] watchdog.Garage                INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.559484776  [2024-12-12 10:34:33] watchdog.Garage                INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.559503049  [2024-12-12 10:34:33] watchdog.Garage                INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.626416710  [2024-12-12 10:34:33] frigate.app                    INFO    : Waiting for capture process for Salon to stop
2024-12-12 09:34:33.626858449  [2024-12-12 10:34:33] watchdog.Salon                 INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.626859641  [2024-12-12 10:34:33] watchdog.Salon                 INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.627214998  [2024-12-12 10:34:33] watchdog.Salon                 INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.627244353  [2024-12-12 10:34:33] watchdog.Salon                 INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.645389677  [2024-12-12 10:34:33] frigate.app                    INFO    : Waiting for capture process for Jardin to stop
2024-12-12 09:34:33.645642903  [2024-12-12 10:34:33] watchdog.Jardin                INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.645671325  [2024-12-12 10:34:33] watchdog.Jardin                INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.646445499  [2024-12-12 10:34:33] watchdog.Jardin                INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.646473692  [2024-12-12 10:34:33] watchdog.Jardin                INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.656208962  [2024-12-12 10:34:33] frigate.app                    INFO    : Waiting for capture process for RAB to stop
2024-12-12 09:34:33.656520567  [2024-12-12 10:34:33] watchdog.RAB                   INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.656522180  [2024-12-12 10:34:33] watchdog.RAB                   INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.657256318  [2024-12-12 10:34:33] watchdog.RAB                   INFO    : Terminating the existing ffmpeg process...
2024-12-12 09:34:33.657290452  [2024-12-12 10:34:33] watchdog.RAB                   INFO    : Waiting for ffmpeg to exit gracefully...
2024-12-12 09:34:33.691682286  [2024-12-12 10:34:33] frigate.app                    INFO    : Waiting for process for GarageDehors to stop

Relevant go2rtc log output

Type
Timestamp
Tag
Message
info
2024-12-12 09:56:36
startup
Preparing new go2rtc config...
info
2024-12-12 09:56:36
startup
Starting go2rtc...
info
2024-12-12 09:56:36
startup
go2rtc platform=linux/amd64 revision=b2399f3 version=1.9.2
info
2024-12-12 09:56:36
startup
config path=/dev/shm/go2rtc.yaml
info
2024-12-12 09:56:36
rtsp
listen addr=:8554
info
2024-12-12 09:56:36
api
listen addr=:1984
info
2024-12-12 09:56:36
webrtc
listen addr=:8555/tcp
info
2024-12-12 09:56:46
startup
Starting go2rtc healthcheck service...

FFprobe output from your camera

docker@docker:~$ ffprobe version 6.1.1-3ubuntu5+esm2 Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.2.0-23ubuntu4)
  configuration: --prefix=/usr --extra-version=3ubuntu5+esm2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[tcp @ 0x6232d81825c0] Port missing in uri
rtsp://acces:X: Invalid argument

My pwd stat with X&

Install method

Docker Compose

Object Detector

CPU (no coral)

Network connection

Wired

Camera make and model

Hikvision mostly

Screenshots of the Frigate UI's System metrics pages

it crashes too quickly

Any other information that may be helpful

vainfo
error: can't connect to X server!
libva info: VA-API version 1.20.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/virtio_gpu_drv_video.so
libva info: Found init function __vaDriverInit_1_20
libva error: /usr/lib/x86_64-linux-gnu/dri/virtio_gpu_drv_video.so init failed
libva info: va_openDriver() returns 2
vaInitialize failed with error code 2 (resource allocation failed),exit

ffmpeg -decoders | grep cuvid
ffmpeg version 6.1.1-3ubuntu5+esm2 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13 (Ubuntu 13.2.0-23ubuntu4)
configuration: --prefix=/usr --extra-version=3ubuntu5+esm2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared
libavutil 58. 29.100 / 58. 29.100
libavcodec 60. 31.102 / 60. 31.102
libavformat 60. 16.100 / 60. 16.100
libavdevice 60. 3.100 / 60. 3.100
libavfilter 9. 12.100 / 9. 12.100
libswscale 7. 5.100 / 7. 5.100
libswresample 4. 12.100 / 4. 12.100
libpostproc 57. 3.100 / 57. 3.100
V..... av1_cuvid Nvidia CUVID AV1 decoder (codec av1)
V..... h264_cuvid Nvidia CUVID H264 decoder (codec h264)
V..... hevc_cuvid Nvidia CUVID HEVC decoder (codec hevc)
V..... mjpeg_cuvid Nvidia CUVID MJPEG decoder (codec mjpeg)
V..... mpeg1_cuvid Nvidia CUVID MPEG1VIDEO decoder (codec mpeg1video)
V..... mpeg2_cuvid Nvidia CUVID MPEG2VIDEO decoder (codec mpeg2video)
V..... mpeg4_cuvid Nvidia CUVID MPEG4 decoder (codec mpeg4)
V..... vc1_cuvid Nvidia CUVID VC1 decoder (codec vc1)
V..... vp8_cuvid Nvidia CUVID VP8 decoder (codec vp8)
V..... vp9_cuvid Nvidia CUVID VP9 decoder (codec vp9)

sudo drm_info -j | jq 'with_entries(.value |= .driver.desc)'
{
"/dev/dri/card1": "virtio GPU"
}

[xabix]

I have added back privileged: true in my compose otherwise the coral doesn't work and this fixed this:
ValueError: Failed to load delegate from libedgetpu.so.1.0

but I still don't get the /dev/dri to work and get this:
2024-12-12 11:00:33.786969330 [2024-12-12 12:00:33] ffmpeg.RAB.detect ERROR : [AVHWDeviceContext @ 0x5cfb0461cd00] libva: /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so init failed

so no sure which device to be pointing to and why none of them worked.

Now that it doesn't crash I get:

On the guest I have:

lspci -k | grep -EA3 'VGA|3D|Display'
00:01.0 VGA compatible controller: Red Hat, Inc. Virtio 1.0 GPU (rev 01)
        Subsystem: Red Hat, Inc. Virtio 1.0 GPU
        Kernel driver in use: virtio-pci
00:1a.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4

On the host I have:

root@Proxmox ~# vainfo
error: can't connect to X server!
libva info: VA-API version 1.17.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_17
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.17 (libva 2.12.0)
vainfo: Driver version: Mesa Gallium driver 22.3.6 for AMD Radeon Graphics (gfx1036, LLVM 15.0.6, DRM 3.58, 6.11.0-1-pve)
vainfo: Supported profile and entrypoints
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSlice
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileAV1Profile0            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

Maybe I need to change this ?
LIBVA_DRIVER_NAME: radeonsi

[xabix]

I got it working but I am not sure that I am using the hwaccel of the GPU for ffmpeg. Maybe :)

I kept:

detectors:
  coral:
    type: edgetpu
    device: usb

ffmpeg:
  #hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 #Enable Hardware Acceleration
  hwaccel_args: "auto"
  global_args: -hide_banner -loglevel error -threads 4
  output_args:
    detect: -threads 4 -f rawvideo -pix_fmt yuv420p
    record: preset-record-generic
  retry_interval: 10