[Detector Support]: Warning log about onnxruntime

[freakytoad1]

Describe the problem you are having

Just recently set up an nvidia 3060 and set my frigate image to tensor-stable. I've passed the GPU through docker and added the following to my frigate config (based on docs):

detectors:
  onnx:
    type: onnx

When the frigate service starts up I see a yellow warning log about onnx. Searched through Github issues and google but didn't see anything about frigate specifically. Is this a message that is important? Is my config incorrect?

2025-10-15 22:24:52.034069759  2025-10-15 22:24:52.034003386 [W:onnxruntime:, transformer_memcpy.cc:74 ApplyImpl] 2 Memcpy nodes are added to the graph main_graph for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.

I use unraid for docker, posted the docker command it runs below.

I've been using my intel iGPU with openvino for a year or so but decided to try out my nvidia card. If theres a sweet spot where I could use both, that would be nice.

Version

0.16.2-4d58206

Frigate config file

mqtt:
  host: <ip>
  user: '{FRIGATE_MQTT_USER}'
  password: '{FRIGATE_MQTT_PASSWORD}'
birdseye:
  enabled: false
record:
  enabled: true
  retain:
    days: 30
    mode: motion
  alerts:
    retain:
      days: 90
  detections:
    retain:
      days: 90
detectors:
  onnx:
    type: onnx
model:
  path: plus://...
ffmpeg:
  hwaccel_args: preset-nvidia
  input_args: preset-rtsp-restream
  output_args:
    record: preset-record-generic
semantic_search:
  enabled: true
  reindex: false
  model_size: large
genai:
  enabled: true
  provider: ollama
  base_url: http://<ip>:11434
  model: llava:7b
detect:
  enabled: true
face_recognition:
  enabled: true
  model_size: large
lpr:
  enabled: True
  device: GPU 
classification:
  bird:
    enabled: true
cameras:
  doorbell:
    ffmpeg:
      inputs:
        # Main Stream
        - path: 
            rtsp://admin:{FRIGATE_RTSP_PASSWORD_DOORBELL}@<ip>:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
          roles:
            - record
        # Sub Stream
        - path: 
            rtsp://admin:{FRIGATE_RTSP_PASSWORD_DOORBELL}@<ip>:554/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif
          roles:
            - detect
    detect:
      width: 768
      height: 576
      fps: 15
      annotation_offset: -1000
    objects:
      track:
        - person
        - face
        - license_plate
        - dog
        - cat
        - car
        - license_plate
        - bicycle
        - amazon
        - fedex
        - ups
        - package
      filters:
        person:
          threshold: 0.85
        dog:
          threshold: 0.85
        cat:
          threshold: 0.85
        bicycle:
          threshold: 0.85
        car:
          threshold: 0.9
    snapshots:
      enabled: true
      required_zones:
        - dw_entrance_doorbell
        - yard_doorbell
        - road_doorbell
        - sidewalk_doorbell
    motion:
      mask: 
        0.999,0.559,0.932,0.57,0.814,0.586,0.708,0.59,0.619,0.592,0.509,0.595,0.429,0.59,0.323,0.588,0.232,0.578,0.115,0.563,0.037,0.541,0,0.517,0,0,1,0
    zones:
      dw_entrance_doorbell:
        coordinates: 0.547,0.629,0.394,0.621,0.311,0.646,0.534,0.657
        inertia: 1
        loitering_time: 0
        objects:
          - car
          - license_plate
      yard_doorbell:
        coordinates: 0.668,0.677,0.47,0.67,0.269,0.65,0,0.661,0,1,1,1,1,0.64
        inertia: 3
        loitering_time: 0
        object:
          - person
          - face
          - dog
          - cat
          - bicycle
      road_doorbell:
        coordinates: 
          0.998,0.565,0.901,0.577,0.811,0.589,0.742,0.593,0.629,0.596,0.523,0.597,0.424,0.593,0.33,0.592,0.276,0.586,0.181,0.577,0.095,0.567,0.04,0.547,0.031,0.563,0.091,0.589,0.181,0.6,0.273,0.612,0.332,0.612,0.393,0.619,0.555,0.623,0.704,0.622,0.822,0.616,0.915,0.607,0.997,0.597
        loitering_time: 0
        objects:
          - bicycle
          - cat
          - dog
          - person
          - face 
      sidewalk_doorbell:
        coordinates: 
          0.997,0.603,0.917,0.607,0.777,0.622,0.641,0.624,0.546,0.629,0.372,0.616,0.274,0.614,0.234,0.635,0.307,0.644,0.537,0.661,0.643,0.664,0.758,0.653,0.86,0.645,0.969,0.634,0.999,0.63
        loitering_time: 0
        objects:
          - cat
          - dog
          - person
          - face
    review:
      alerts:
        required_zones:
          - dw_entrance_doorbell
          - yard_doorbell
  garage:
    ffmpeg:
      inputs:
        # Main Stream
        - path: 
            rtsp://admin:{FRIGATE_RTSP_PASSWORD_GARAGE}@<ip>2:554/Streaming/Channels/101?transportmode=unicast&profile=Profile_1
          roles:
            - record
        # Sub Stream
        - path: 
            rtsp://admin:{FRIGATE_RTSP_PASSWORD_GARAGE}@<ip>:554/Streaming/Channels/102?transportmode=unicast&profile=Profile_2
          roles:
            - detect
    detect:
      width: 856
      height: 480
      fps: 6
      annotation_offset: -1000
    motion:
      threshold: 50
      contour_area: 30
      mask:
        - 0,0,856,0,856,111,202,88,0,199
    objects:
      track:
        - person
        - face
      filters:
        person:
          threshold: 0.9
    snapshots:
      enabled: true
  back:
    ffmpeg:
      inputs:
        # Main Stream
        - path: 
            rtsp://admin:{FRIGATE_RTSP_PASSWORD_BACK_DUO}@<ip>:554/Preview_01_main
          roles:
            - record
        # Sub Stream
        - path: 
            rtsp://admin:{FRIGATE_RTSP_PASSWORD_BACK_DUO}@<ip>:554/Preview_01_sub
          roles:
            - detect
    motion:
      threshold: 50
      contour_area: 30
      mask: 
        0.038,0.701,0.143,0.529,0.288,0.225,0.323,0.113,0.475,0.065,0.611,0.104,0.677,0.243,0.807,0.557,0.952,0.821,1,0.65,1,0,0,0,0,0.516
    detect:
      width: 1536
      height: 432
      fps: 7

      annotation_offset: -1000
    objects:
      track:
        - person
        - face
        - cat
        - dog
      filters:
        person:
          threshold: 0.9
          mask:
            - 416,267,558,257,510,96,384,117
        dog:
          threshold: 0.8
        cat:
          threshold: 0.8
    snapshots:
      enabled: true
  front:
    ffmpeg:
      inputs:
        # Main Stream
        - path: 
            rtsp://admin:{FRIGATE_RTSP_PASSWORD_FRONT}@<ip>:554/Streaming/Channels/101?transportmode=unicast&profile=Profile_1
          roles:
            - record
        # Sub Stream
        - path: 
            rtsp://admin:{FRIGATE_RTSP_PASSWORD_FRONT}@<ip>:554/Streaming/Channels/102?transportmode=unicast&profile=Profile_2
          roles:
            - detect
    detect:
      width: 856
      height: 480
      fps: 6
      annotation_offset: -1000
    objects:
      track:
        - person
        - face
        - dog
        - cat
        - car
        - license_plate
        - bicycle
      filters:
        person:
          threshold: 0.85
        dog:
          threshold: 0.85
        cat:
          threshold: 0.85
        bicycle:
          threshold: 0.85
        car:
          threshold: 0.9
    snapshots:
      enabled: true
      required_zones:
        - yard_front
        - road
        - dw_entrance_front
        - sidewalk
    zones:
      dw_entrance_front:
        coordinates: 0.013,0.417,0.021,0.52,0.31,0.421,0.214,0.341
        loitering_time: 0
        interia: 1
        objects:
          - car
          - license_plate
      yard_front:
        coordinates: 
          0,0.547,0,1,1,1,1,0.382,0.962,0.357,0.865,0.307,0.756,0.322,0.599,0.353,0.508,0.376,0.425,0.398,0.328,0.43
        loitering_time: 0
        object:
          - person
          - face
          - dog
          - cat
          - bicycle
        inertia: 3
        objects:
          - person
          - face
          - dog
          - cat
          - bicycle
      road:
        coordinates: 
          0,0.313,0.001,0.425,0.222,0.337,0.306,0.302,0.353,0.296,0.468,0.27,0.628,0.248,0.751,0.242,0.86,0.242,0.954,0.25,1,0.258,1,0.237,0.831,0.189,0.622,0.176,0.396,0.19,0.294,0.197,0.139,0.25
        loitering_time: 0
        interia: 1
        objects:
          - person
          - face
          - dog
          - cat
          - bicycle
      sidewalk:
        coordinates: 
          0,0.522,0.075,0.499,0.331,0.405,0.483,0.361,0.583,0.341,0.741,0.303,0.872,0.295,1,0.379,1,0.257,0.868,0.245,0.724,0.245,0.606,0.247,0.436,0.279,0.3,0.306,0.221,0.337,0.112,0.379,0,0.424
        loitering_time: 0
        objects:
          - person
          - face
          - dog
          - cat
          - bicycle
    review:
      alerts:
        required_zones:
          - yard_front
          - dw_entrance_front
    motion:
      threshold: 50
      contour_area: 30
      improve_contrast: true
      mask:
        - 0.835,0.893,0.938,0.893,0.938,0.93,0.835,0.929
        - 0,0.314,0.023,0.3,0.099,0.268,0.214,0.226,0.321,0.195,0.436,0.183,0.579,0.176,0.63,0.174,0.71,0.179,0.794,0.188,0.828,0.191,0.866,0.203,0.894,0.211,0.951,0.225,1,0.235,1,0,0,0
version: 0.16-0

docker-compose file or Docker CLI command

docker run
  -d
  --name='frigate'
  --net='xxx'
  --ip='xxx'
  --pids-limit 2048
  --privileged=true
  -e TZ="America/Chicago"
  -e HOST_OS="Unraid"
  -e HOST_HOSTNAME="unraid"
  -e HOST_CONTAINERNAME="frigate"
  -e 'TCP_PORT_5000'='5000'
  -e 'TCP_PORT_8554'='8554'
  -e 'FRIGATE_RTSP_PASSWORD_DOORBELL'='xxx'
  -e 'PLUS_API_KEY'='xxx'
  -e 'TCP_PORT_8555'='8555'
  -e 'UDP_PORT_8555'='8555'
  -e 'TCP_PORT_1984'='1984'
  -e 'FRIGATE_MQTT_USER'='xxx'
  -e 'FRIGATE_MQTT_PASSWORD'='xxx'
  -e 'FRIGATE_RTSP_PASSWORD_GARAGE'='xxx'
  -e 'FRIGATE_RTSP_PASSWORD_BACK'='xxx'
  -e 'FRIGATE_RTSP_PASSWORD_BACK_DUO'='xxx'
  -e 'FRIGATE_RTSP_PASSWORD_FRONT'='xxx'
  -e 'YOLO_MODELS'=''
  -e 'NVIDIA_DRIVER_CAPABILITIES'='all'
  -e 'NVIDIA_VISIBLE_DEVICES'='GPU-xxx'
  -l net.unraid.docker.managed=dockerman
  -l net.unraid.docker.webui='http://[IP]:[PORT:5000]'
  -l net.unraid.docker.icon='https://raw.githubusercontent.com/yayitazale/unraid-templates/main/frigate.png'
  -v '/mnt/cache/appdata/frigate':'/config':'rw'
  -v '/mnt/user/frigate/':'/media/frigate':'rw'
  -v '/etc/localtime':'/etc/localtime':'rw'
  --gpus=all
  --shm-size=512mb
  --mount type=tmpfs,target=/tmp/cache,tmpfs-size=1000000000
  --restart unless-stopped
  --memory=8G
  --cpu-shares=2
  --device=/dev/dri
  --runtime=nvidia 'ghcr.io/blakeblackshear/frigate:stable-tensorrt'

Relevant Frigate log output

2025-10-15 22:33:54.149817573  [2025-10-15 22:33:54] frigate.api.fastapi_app        INFO    : Starting FastAPI app
2025-10-15 22:33:54.398187114  [2025-10-15 22:33:54] frigate.api.fastapi_app        INFO    : FastAPI started
2025-10-15 22:33:54.670514708  127.0.0.1 - - [15/Oct/2025:22:33:54 -0500] "" 400 0 "-" "-" "-"
2025-10-15 22:33:55.521932662  2025-10-15 22:33:55.521881264 [W:onnxruntime:, transformer_memcpy.cc:74 ApplyImpl] 2 Memcpy nodes are added to the graph main_graph for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.
2025-10-15 22:33:55.557506597  loading data from : /config/model_cache/facedet/landmarkdet.yaml
2025-10-15 22:33:56.883372392  INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
2025-10-15 22:33:58.984626790  [INFO] Starting go2rtc healthcheck service...

Install method

Docker CLI

Object Detector

TensorRT

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[inkeep-ai-assistant[bot]]

The warning message you're seeing is normal and not a cause for concern (1). This warning appears when using ONNX with TensorRT and indicates that some memory copy operations are being added to the computation graph, which might have a minor performance impact but doesn't affect functionality (1).

Your configuration is correct. When using the -tensorrt Frigate image with ONNX, TensorRT will automatically be detected and used as a detector when a supported ONNX model is configured (2). The fact that you're seeing GPU usage in your system metrics (as shown in your screenshots) confirms that the GPU is being utilized for inference, which indicates the TensorRT execution provider is active (1).

For ONNX with Nvidia GPUs, Nvidia GPUs will automatically be detected and used with the ONNX detector in the -tensorrt Frigate image (2). The automatic detection means you don't need additional configuration beyond what you have.

Regarding using both your Intel iGPU and Nvidia GPU simultaneously: multiple detectors can be defined assuming GPU resources are available (2). You could potentially configure both OpenVINO and ONNX detectors, but note that multiple detectors cannot be mixed for object detection (ex: OpenVINO and Coral EdgeTPU cannot be used for object detection at the same time) (3). This limitation applies to the primary object detection task, though it doesn't affect using hardware for accelerating other tasks such as semantic search (3).

📚 Sources:

1. [Detector Support]: How do I know if onnx is correctly using TensorRT?
2. Supported Hardware
3. Supported Hardware

---

Have another question? Just tag @inkeep.