[HW Accel Support]: Trying to generate models for TensorRT

[b-rad15]

Describe the problem you are having

While trying to setup frigate with tensorrt inside podman I run into this issue below of a missing library while generating the models. /usr/local/lib/python3.9/dist-packages/nvidia/cudnn/lib/libcudnn_cnn_infer.so.8 exists and symlinking it next to libcuda.so.1 doesn't help. Is there something that I was supposed to install on the host to provide this? The host has a GTX 1070 that has a CDI setup that works for workloads in other containers and is detected by nvidia-smi inside the frigate container. #5015 seems like a similar issue but was fixed a while ago. The image I'm using is ghcr.io/blakeblackshear/frigate:stable-tensorrt.

Version

Docker Image ID is 3fff86f943b6 (UI does not load to check there)

Frigate config file

# yaml-language-server: $schema=http://frigate_host:5000/api/config/schema.json
mqtt:
  enabled: False

go2rtc:
  streams:
    esp32-front-yard:
      - "ffmpeg:http://192.168.1.217/stream#video=h264#hardware"

cameras:
  esp32-front-yard: # <------ Name the camera
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
        - path: rtsp://127.0.0.1:8554/esp32-front-yard # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record
    detect:
      enabled: True # <---- disable detection until you have a working camera feed
      width: 1600 # <---- update for your camera's resolution
      height: 1200 # <---- update for your camera's resolution

detectors:
  tensorrt:
    type: tensorrt
    device: 0 #This is the default, select the first GPU

model:
  path: /config/model_cache/tensorrt/yolov4-tiny-416.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 416
  height: 416

logger:
  default: debug

docker-compose file or Docker CLI command

# Quadlet Unit File
[Unit]
Description=Frigate Camera NVR System Podman+Nvidia

[Container]
AddDevice=nvidia.com/gpu=all
ContainerName=frigate
Environment=FRIGATE_RTSP_PASSWORD=******* YOLO_MODELS=yolov4-tiny-416 USE_FP16=false
Image=ghcr.io/blakeblackshear/frigate:stable-tensorrt
Mount=type=tmpfs,destination=/tmp/cache,tmpfs-size=4000000000
PublishPort=5001:5000
PublishPort=8554:8554
PublishPort=8555:8555/tcp
PublishPort=8555:8555/udp
ShmSize=1g
Volume=******/frigate-data/frigate:/media/frigate
Volume=******/frigate-data/config:/config
Volume=/etc/localtime:/etc/localtime:ro

[Service]
Restart=always

[Install]
WantedBy=default.target

Relevant log output

Creating yolov4-608.cfg and yolov4-608.weights
Creating yolov7x-640.cfg and yolov7x-640.weights

Done.
2024-05-10 00:42:43.012304947  [INFO] Starting go2rtc healthcheck service...

Generating yolov4-608.trt. This may take a few minutes.

/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:147: DeprecationWarning: Use network created with NetworkDefinitionCreationFlag::EXPLICIT_BATCH flag instead.
  builder.max_batch_size = MAX_BATCH_SIZE
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:149: DeprecationWarning: Use set_memory_pool_limit instead.
  config.max_workspace_size = 1 << 30
/usr/local/src/tensorrt_demos/yolo/onnx_to_tensorrt.py:172: DeprecationWarning: Use build_serialized_network instead.
  engine = builder.build_engine(network, config)
Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory
/etc/s6-overlay/s6-rc.d/trt-model-prepare/run: line 82:   177 Aborted                 (core dumped) $cmd > /tmp/onnx_to_tensorrt.log

FFprobe output from your camera

N/A

Operating system

Other Linux

Install method

Docker Compose

Network connection

Mixed

Camera make and model

N/A

Any other information that may be helpful

No response

[NickM-27]

this is usually due to missing libraries on host or incorrectly installed nvidia docker toolkit