[HW Accel Support]: 2 Gpu's 1 Frigate #13693

mahchet · 2024-09-12T08:50:17Z

mahchet
Sep 12, 2024

Describe the problem you are having

Hello everyone, I have a need to use two hardware accelerators.
I managed to encode video using one GPU, NVIDIA GeForce GTX 1050, and now I have added an NVIDIA GeForce GTX 1660.
However, only the NVIDIA GeForce GTX 1660 is working, while the NVIDIA GeForce GTX 1050 remains idle.
Could you advise me on how to direct RTSP streams to specific GPUs in my configuration? I couldn’t find the appropriate documentation on this, and I have about 60 RTSP streams.

Version

0.14.1-f4f3cfa

In which browser(s) are you experiencing the issue with?

No response

Frigate config file

mqtt:
  enabled: false
detect:
  enabled: false
database:
  path: /config/frigate.db
record:
  enabled: true
  retain:
    days: 1
logger:
  logs:
    frigate.record.maintainer: debug
ffmpeg:
  hwaccel_args: preset-nvidia-h264
cameras:
  cam40:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam41:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam42:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam44:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam45:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam46:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam47:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam48:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam49:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam50:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam51:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam52:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam53:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam54:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam55:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam56:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam57:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam58:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam60:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam63:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam64:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam66:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam67:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam68:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam70:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam71:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam73:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam74:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam77:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam78:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam79:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam81:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam83:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam84:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam85:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam86:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam87:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam89:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam90:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam91:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam92:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam93:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam94:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam95:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam96:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam97:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam98:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam99:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam100:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam102:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam105:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam107:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam108:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam110:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    privileged: true # this may not be necessary for all setups
    image: ghcr.io/blakeblackshear/frigate:stable
    deploy:    # <------------- Add this section
      resources:
        reservations:
         devices:
            - driver: nvidia
              count: 2 # number of GPU3
              capabilities: [gpu]
    shm_size: "1000MB" # update for your cameras based on calculation above
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config
      - ./storage:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 10000000000
    devices:
      - /dev/nvidia-caps:/dev/nvidia-caps
      - /dev/nvidia0:/dev/nvidia0
      - /dev/nvidiactl:/dev/nvidiactl
      - /dev/nvidia-modeset:/dev/nvidia-modeset
      - /dev/nvidia-uvm:/dev/nvidia-uvm
      - /dev/nvidia-uvm-tools:/dev/nvidia-uvm-tools
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "S4fopae8"
    networks:
      - internet
    restart: unless-stopped

networks:
  internet:
    external: true

Relevant Frigate log output

2024-09-12 11:36:37
ffmpeg.cam48.detect
[h264 @ 0x561acb21e840] decoder->cvdl->cuvidCreateDecoder(&decoder->decoder, params) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam92.detect
[h264 @ 0x55aedde0bf40] Failed setup for format cuda: hwaccel initialisation returned error.
Error
2024-09-12 11:36:37
ffmpeg.cam48.detect
[h264 @ 0x561acb21e840] Failed setup for format cuda: hwaccel initialisation returned error.
Error
2024-09-12 11:36:37
ffmpeg.cam48.detect
Impossible to convert between the formats supported by the filter 'Parsed_fps_0' and the filter 'auto_scale_0'
Error
2024-09-12 11:36:37
ffmpeg.cam92.detect
Impossible to convert between the formats supported by the filter 'Parsed_fps_0' and the filter 'auto_scale_0'
Error
2024-09-12 11:36:37
ffmpeg.cam48.detect
Error reinitializing filters!
Error
2024-09-12 11:36:37
ffmpeg.cam92.detect
Error reinitializing filters!
Error
2024-09-12 11:36:37
ffmpeg.cam48.detect
Failed to inject frame into filter network: Function not implemented
Error
2024-09-12 11:36:37
ffmpeg.cam92.detect
Failed to inject frame into filter network: Function not implemented
Error
2024-09-12 11:36:37
ffmpeg.cam48.detect
Error while processing the decoded data for stream #0:0
Error
2024-09-12 11:36:37
ffmpeg.cam92.detect
Error while processing the decoded data for stream #0:0
Error
2024-09-12 11:36:37
frigate.video
cam50: Unable to read frames from ffmpeg process.
Error
2024-09-12 11:36:37
frigate.video
cam50: ffmpeg process is not running. exiting capture thread...
Error
2024-09-12 11:36:37
frigate.video
cam46: Unable to read frames from ffmpeg process.
Error
2024-09-12 11:36:37
frigate.video
cam46: ffmpeg process is not running. exiting capture thread...
Error
2024-09-12 11:36:37
watchdog.cam58
Ffmpeg process crashed unexpectedly for cam58.
Error
2024-09-12 11:36:37
watchdog.cam58
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam58.detect
[AVHWDeviceContext @ 0x55565f159400] cu->cuCtxCreate(&hwctx->cuda_ctx, desired_flags, hwctx->internal->cuda_device) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam58.detect
Device creation failed: -542398533.
Error
2024-09-12 11:36:37
ffmpeg.cam58.detect
[h264 @ 0x55565f1608c0] No device available for decoder: device type cuda needed for codec h264.
Error
2024-09-12 11:36:37
ffmpeg.cam58.detect
Device setup failed for decoder on input stream #0:0 : Generic error in an external library
Error
2024-09-12 11:36:37
watchdog.cam51
Ffmpeg process crashed unexpectedly for cam51.
Error
2024-09-12 11:36:37
watchdog.cam51
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam51.detect
[AVHWDeviceContext @ 0x5594f85c5180] cu->cuCtxCreate(&hwctx->cuda_ctx, desired_flags, hwctx->internal->cuda_device) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam51.detect
Device creation failed: -542398533.
Error
2024-09-12 11:36:37
ffmpeg.cam51.detect
[h264 @ 0x5594f85c8540] No device available for decoder: device type cuda needed for codec h264.
Error
2024-09-12 11:36:37
ffmpeg.cam51.detect
Device setup failed for decoder on input stream #0:0 : Generic error in an external library
Error
2024-09-12 11:36:37
watchdog.cam56
Ffmpeg process crashed unexpectedly for cam56.
Error
2024-09-12 11:36:37
watchdog.cam56
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam56.detect
[segment @ 0x55cd7b9f04c0] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
Error
2024-09-12 11:36:37
ffmpeg.cam56.detect
[h264 @ 0x55cd7c77a780] decoder->cvdl->cuvidCreateDecoder(&decoder->decoder, params) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam56.detect
[h264 @ 0x55cd7c77a780] Failed setup for format cuda: hwaccel initialisation returned error.
Error
2024-09-12 11:36:37
ffmpeg.cam56.detect
Impossible to convert between the formats supported by the filter 'Parsed_fps_0' and the filter 'auto_scale_0'
Error
2024-09-12 11:36:37
ffmpeg.cam56.detect
Error reinitializing filters!
Error
2024-09-12 11:36:37
ffmpeg.cam56.detect
Failed to inject frame into filter network: Function not implemented
Error
2024-09-12 11:36:37
watchdog.cam57
Ffmpeg process crashed unexpectedly for cam57.
Error
2024-09-12 11:36:37
watchdog.cam49
Ffmpeg process crashed unexpectedly for cam49.
Error
2024-09-12 11:36:37
watchdog.cam63
Ffmpeg process crashed unexpectedly for cam63.
Error
2024-09-12 11:36:37
watchdog.cam63
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam63.detect
Guessed Channel Layout for Input Stream #0.1 : mono
Error
2024-09-12 11:36:37
ffmpeg.cam63.detect
[AVHWDeviceContext @ 0x555cb83edd00] cu->cuCtxCreate(&hwctx->cuda_ctx, desired_flags, hwctx->internal->cuda_device) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam63.detect
Device creation failed: -542398533.
Error
2024-09-12 11:36:37
ffmpeg.cam63.detect
[h264 @ 0x555cb8364800] No device available for decoder: device type cuda needed for codec h264.
Error
2024-09-12 11:36:37
ffmpeg.cam63.detect
Device setup failed for decoder on input stream #0:0 : Generic error in an external library
Error
2024-09-12 11:36:37
watchdog.cam78
Ffmpeg process crashed unexpectedly for cam78.
Error
2024-09-12 11:36:37
watchdog.cam78
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam78.detect
[segment @ 0x557d9b2ddc80] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
Error
2024-09-12 11:36:37
ffmpeg.cam78.detect
[h264 @ 0x557d9bbca880] decoder->cvdl->cuvidCreateDecoder(&decoder->decoder, params) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam78.detect
[h264 @ 0x557d9bbca880] Failed setup for format cuda: hwaccel initialisation returned error.
Error
2024-09-12 11:36:37
ffmpeg.cam78.detect
Impossible to convert between the formats supported by the filter 'Parsed_fps_0' and the filter 'auto_scale_0'
Error
2024-09-12 11:36:37
ffmpeg.cam78.detect
Error reinitializing filters!
Error
2024-09-12 11:36:37
ffmpeg.cam78.detect
Failed to inject frame into filter network: Function not implemented
Error
2024-09-12 11:36:37
ffmpeg.cam78.detect
Error while processing the decoded data for stream #0:0
Error
2024-09-12 11:36:37
ffmpeg.cam56.detect
Error while processing the decoded data for stream #0:0
Error
2024-09-12 11:36:37
watchdog.cam57
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam57.detect
[AVHWDeviceContext @ 0x55b47cb04dc0] cu->cuCtxCreate(&hwctx->cuda_ctx, desired_flags, hwctx->internal->cuda_device) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam57.detect
Device creation failed: -542398533.
Error
2024-09-12 11:36:37
watchdog.cam64
Ffmpeg process crashed unexpectedly for cam64.
Error
2024-09-12 11:36:37
watchdog.cam107
Ffmpeg process crashed unexpectedly for cam107.
Error
2024-09-12 11:36:37
watchdog.cam49
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
watchdog.cam84
Ffmpeg process crashed unexpectedly for cam84.
Error
2024-09-12 11:36:37
ffmpeg.cam57.detect
[h264 @ 0x55b47c9f25c0] No device available for decoder: device type cuda needed for codec h264.
Error
2024-09-12 11:36:37
watchdog.cam73
Ffmpeg process crashed unexpectedly for cam73.
Error
2024-09-12 11:36:37
watchdog.cam68
Ffmpeg process crashed unexpectedly for cam68.
Error
2024-09-12 11:36:37
ffmpeg.cam49.detect
Guessed Channel Layout for Input Stream #0.1 : mono
Error
2024-09-12 11:36:37
watchdog.cam68
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
watchdog.cam64
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam57.detect
Device setup failed for decoder on input stream #0:0 : Generic error in an external library
Error
2024-09-12 11:36:37
watchdog.cam84
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam49.detect
[AVHWDeviceContext @ 0x562443db0100] cu->cuCtxCreate(&hwctx->cuda_ctx, desired_flags, hwctx->internal->cuda_device) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam68.detect
[AVHWDeviceContext @ 0x563664123e40] cu->cuCtxCreate(&hwctx->cuda_ctx, desired_flags, hwctx->internal->cuda_device) failed -> CUDA_ERROR_UNKNOWN: unknown error
Error
2024-09-12 11:36:37
watchdog.cam107
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam68.detect
Device creation failed: -542398533.
Error
2024-09-12 11:36:37
ffmpeg.cam49.detect
Device creation failed: -542398533.
Error
2024-09-12 11:36:37
ffmpeg.cam84.detect
[AVHWDeviceContext @ 0x55b92741ae40] cu->cuCtxCreate(&hwctx->cuda_ctx, desired_flags, hwctx->internal->cuda_device) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam68.detect
[h264 @ 0x563664137800] No device available for decoder: device type cuda needed for codec h264.
Error
2024-09-12 11:36:37
ffmpeg.cam49.detect
[h264 @ 0x562443db5540] No device available for decoder: device type cuda needed for codec h264.
Error
2024-09-12 11:36:37
ffmpeg.cam64.detect
Guessed Channel Layout for Input Stream #0.1 : mono
Error
2024-09-12 11:36:37
ffmpeg.cam68.detect
Device setup failed for decoder on input stream #0:0 : Generic error in an external library
Error
2024-09-12 11:36:37
ffmpeg.cam84.detect
Device creation failed: -542398533.
Error
2024-09-12 11:36:37
ffmpeg.cam49.detect
Device setup failed for decoder on input stream #0:0 : Generic error in an external library
Error
2024-09-12 11:36:37
watchdog.cam73
The following ffmpeg logs include the last 100 lines prior to exit.
Error
2024-09-12 11:36:37
ffmpeg.cam107.detect
[segment @ 0x55d770d39b00] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
Error
2024-09-12 11:36:37
ffmpeg.cam64.detect
[mp4 @ 0x556fb0778dc0] Could not find tag for codec pcm_mulaw in stream #1, codec not currently supported in container
Error
2024-09-12 11:36:37
ffmpeg.cam84.detect
[h264 @ 0x55b92730dcc0] No device available for decoder: device type cuda needed for codec h264.
Error
2024-09-12 11:36:37
ffmpeg.cam107.detect
[h264 @ 0x55d771c84300] decoder->cvdl->cuvidCreateDecoder(&decoder->decoder, params) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam84.detect
Device setup failed for decoder on input stream #0:0 : Generic error in an external library
Error
2024-09-12 11:36:37
ffmpeg.cam64.detect
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error
2024-09-12 11:36:37
ffmpeg.cam73.detect
[AVHWDeviceContext @ 0x558752da04c0] cu->cuCtxCreate(&hwctx->cuda_ctx, desired_flags, hwctx->internal->cuda_device) failed -> CUDA_ERROR_OUT_OF_MEMORY: out of memory
Error
2024-09-12 11:36:37
ffmpeg.cam107.detect
[h264 @ 0x55d771c84300] Failed setup for format cuda: hwaccel initialisation returned error.
Error
2024-09-12 11:36:37
ffmpeg.cam64.detect
Error initializing output stream 0:1 --
Error
2024-09-12 11:36:37
ffmpeg.cam73.detect
Device creation failed: -542398533.
Error
2024-09-12 11:36:37
ffmpeg.cam107.detect
Impossible to convert between the formats supported by the filter 'Parsed_fps_0' and the filter 'auto_scale_0'
Error
2024-09-12 11:36:37
ffmpeg.cam64.detect
Error
2024-09-12 11:36:37
ffmpeg.cam73.detect
[h264 @ 0x558752da2f80] No device available for decoder: device type cuda needed for codec h264.
Error
2024-09-12 11:36:37
ffmpeg.cam107.detect
Error reinitializing filters!
Error
2024-09-12 11:36:37
ffmpeg.cam73.detect
Device setup failed for decoder on input stream #0:0 : Generic error in an external library
Error
2024-09-12 11:36:37
ffmpeg.cam107.detect
Failed to inject frame into filter network: Function not implemented
Error
2024-09-12 11:36:37
ffmpeg.cam107.detect
Error while processing the decoded data for stream #0:0

Relevant go2rtc log output

Info
2024-09-12 11:28:32
startup
Preparing new go2rtc config...
Info
2024-09-12 11:28:32
startup
Starting go2rtc...
Info
2024-09-12 11:28:32
startup
go2rtc platform=linux/amd64 revision=b2399f3 version=1.9.2
Info
2024-09-12 11:28:32
startup
config path=/dev/shm/go2rtc.yaml
Info
2024-09-12 11:28:32
rtsp
listen addr=:8554
Info
2024-09-12 11:28:32
api
listen addr=:1984
Info
2024-09-12 11:28:32
webrtc
listen addr=:8555/tcp
Info
2024-09-12 11:28:42
startup
Starting go2rtc healthcheck service...

FFprobe output from your camera

Input #0, rtsp, from 'rtsp://puk:[email protected]/axis-media/media.amp':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.040000, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuvj420p(pc, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25.08 tbr, 90k tbn

Operating system

Other Linux

Install method

Docker Compose

Network connection

Wired

Camera make and model

hikvision

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

root@6c7d2bef6eb5:/opt/frigate# nvidia-smi
Thu Sep 12 11:42:03 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.107.02             Driver Version: 550.107.02     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1660 ...    On  |   00000000:01:00.0 Off |                  N/A |
| 42%   39C    P2             46W /  125W |    5826MiB /   6144MiB |     11%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce GTX 1050        On  |   00000000:06:00.0 Off |                  N/A |
| 33%   24C    P8             N/A /   75W |       5MiB /   2048MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+

root@6c7d2bef6eb5:/opt/frigate# ls -la /dev/nvidia*
crw-rw-rw- 1 root sudo 195, 254 Sep 12 11:28 /dev/nvidia-modeset
crw-rw-rw- 1 root root 509,   0 Sep 11 17:33 /dev/nvidia-uvm
crw-rw-rw- 1 root root 509,   1 Sep 11 17:33 /dev/nvidia-uvm-tools
crw-rw-rw- 1 root sudo 195,   0 Sep 11 17:32 /dev/nvidia0
crw-rw-rw- 1 root sudo 195,   1 Sep 11 17:33 /dev/nvidia1
crw-rw-rw- 1 root sudo 195, 255 Sep 11 17:32 /dev/nvidiactl

/dev/nvidia-caps:
total 0
drwxr-xr-x 2 root root     80 Sep 12 11:28 .
drwxr-xr-x 7 root root    500 Sep 12 11:28 ..
cr-------- 1 root root 234, 1 Sep 12 11:28 nvidia-cap1
cr--r--r-- 1 root root 234, 2 Sep 12 11:28 nvidia-cap2

Answered by NickM-27

Sep 16, 2024

mqtt:
  enabled: false
detect:
  enabled: false
database:
  path: /config/frigate.db
record:
  enabled: true
  retain:
    days: 1
logger:
  logs:
    frigate.record.maintainer: debug
ffmpeg:
  hwaccel_args: preset-nvidia-h264
cameras:
  cam98:
    ffmpeg:
      global_args: -hide_banner -loglevel warning -threads 2 -hwaccel_device 0
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam97:
    ffmpeg:
      global_args: -hide_banner -loglevel warning -threads 2 -hwaccel_device 1
      inputs:
       - path: rtsp://puk:[email protected]/axis-media/media.amp
         input_args: preset…

View full answer

NickM-27 · 2024-09-12T13:15:48Z

NickM-27
Sep 12, 2024
Collaborator Sponsor

See #7792 (reply in thread)

0 replies

mahchet · 2024-09-13T08:11:44Z

mahchet
Sep 13, 2024
Author

I tried to set it up for two cameras first, but it's still writing to only one. Maybe I made a mistake somewhere?

gpu frigate # cat config/config.yaml
mqtt:
  enabled: false
detect:
  enabled: false
database:
  path: /config/frigate.db
record:
  enabled: true
  retain:
    days: 1
logger:
  logs:
    frigate.record.maintainer: debug
go2rtc:
#  log:
#    exec: trace
  ffmpeg:
    cuda0: "-fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -hwaccel_device 0 -rtsp_transport tcp -i {input}"
    cuda1: "-fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -hwaccel_device 1 -rtsp_transport tcp -i {input}"
  streams:
    cam98:
      - ffmpeg:rtsp://puk:[email protected]/axis-media/media.amp
    cam98_sub:
      - "ffmpeg:cam98#input=cuda1#video=h264#width=1920#height=1080#raw=-fpsmax 10 -gpu 1#hardware=cuda"
    cam103:
      - ffmpeg:rtsp://puk:[email protected]/axis-media/media.amp
    cam103_sub:
      - "ffmpeg:cam103#input=cuda0#video=h264#width=1920#height=1080#raw=-fpsmax 10 -gpu 0#hardware=cuda"
cameras:
  cam98:
    ffmpeg:
      output_args:
        record: preset-record-generic
      inputs:
        - path: rtsp://127.0.0.1:8554/cam98
          input_args: preset-rtsp-generic
          roles:
            - record
  cam103:
    ffmpeg:
      output_args:
        record: preset-record-generic
      inputs:
        - path: rtsp://127.0.0.1:8554/cam103
          input_args: preset-rtsp-generic
          roles:
            - record

version: 0.14

gpu frigate # nvidia-smi 
Fri Sep 13 11:08:30 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.107.02             Driver Version: 550.107.02     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1660 ...    On  |   00000000:01:00.0 Off |                  N/A |
| 43%   33C    P2             34W /  125W |     345MiB /   6144MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce GTX 1050        On  |   00000000:06:00.0 Off |                  N/A |
| 33%   23C    P8             N/A /   75W |       5MiB /   2048MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+

7 replies

mahchet Sep 16, 2024
Author

@NickM-27
thank you for answer,I solved my problem by moving the settings directly into Frigate without using go2rtc. Maybe my config will help people who need to parallelize across two different NVIDIA GPUs.

cameras:
  cam98:
    ffmpeg:
      hwaccel_args: ["-hwaccel", "cuda", "-hwaccel_device", "1"]
      inputs:
        - path: rtsp://ouk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam97:
    ffmpeg:
      hwaccel_args: ["-hwaccel", "cuda", "-hwaccel_device", "0"]
      inputs:
       - path: rtsp://puk:[email protected]/axis-media/media.amp
         input_args: preset-rtsp-restream
         roles:
           - record

NickM-27 Sep 16, 2024
Collaborator Sponsor

you can get higher efficiency by adding the -hwaccel_device 1 arg to global_args and use preset-nvidia preset

mahchet Sep 16, 2024
Author

@NickM-27 i'm newbie and don't understand where in my config i must get it, could you help me ?

mqtt:
  enabled: false
detect:
  enabled: false
database:
  path: /config/frigate.db
record:
  enabled: true
  retain:
    days: 1
logger:
  logs:
    frigate.record.maintainer: debug
ffmpeg:
  hwaccel_args: preset-nvidia-h264
cameras:
  cam98:
    ffmpeg:
      hwaccel_args: ["-hwaccel", "cuda", "-hwaccel_device", "1"]
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam97:
    ffmpeg:
      hwaccel_args: ["-hwaccel", "cuda", "-hwaccel_device", "0"]
      inputs:
       - path: rtsp://puk:[email protected]/axis-media/media.amp
         input_args: preset-rtsp-restream
         roles:
           - record

NickM-27 Sep 16, 2024
Collaborator Sponsor

mqtt:
  enabled: false
detect:
  enabled: false
database:
  path: /config/frigate.db
record:
  enabled: true
  retain:
    days: 1
logger:
  logs:
    frigate.record.maintainer: debug
ffmpeg:
  hwaccel_args: preset-nvidia-h264
cameras:
  cam98:
    ffmpeg:
      global_args: -hide_banner -loglevel warning -threads 2 -hwaccel_device 0
      inputs:
        - path: rtsp://puk:[email protected]/axis-media/media.amp
          input_args: preset-rtsp-restream
          roles:
            - record
  cam97:
    ffmpeg:
      global_args: -hide_banner -loglevel warning -threads 2 -hwaccel_device 1
      inputs:
       - path: rtsp://puk:[email protected]/axis-media/media.amp
         input_args: preset-rtsp-restream
         roles:
           - record

Answer selected by mahchet

mahchet Sep 16, 2024
Author

@NickM-27 Thank you very much, you are right, here is the GPU-utilization with my settings:

gpu ~ # nvidia-smi
Mon Sep 16 19:58:09 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.107.02             Driver Version: 550.107.02     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1660 ...    On  |   00000000:01:00.0 Off |                  N/A |
| 42%   33C    P2             34W /  125W |     345MiB /   6144MiB |      2%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce GTX 1050        On  |   00000000:06:00.0 Off |                  N/A |
| 33%   34C    P0             N/A /   75W |     293MiB /   2048MiB |     21%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+

and it's with yours:

gpu ~ # nvidia-smi
Mon Sep 16 20:04:01 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.107.02             Driver Version: 550.107.02     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1660 ...    On  |   00000000:01:00.0 Off |                  N/A |
| 42%   34C    P2             34W /  125W |     345MiB /   6144MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce GTX 1050        On  |   00000000:06:00.0 Off |                  N/A |
| 33%   35C    P0             N/A /   75W |     293MiB /   2048MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+

maybe describe this in your frigat documentation ?

Uh oh!

[HW Accel Support]: 2 Gpu's 1 Frigate #13693

Uh oh!

mahchet Sep 12, 2024

Describe the problem you are having

Version

In which browser(s) are you experiencing the issue with?

Frigate config file

docker-compose file or Docker CLI command

Relevant Frigate log output

Relevant go2rtc log output

FFprobe output from your camera

Operating system

Install method

Network connection

Camera make and model

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

Replies: 2 comments · 7 replies

Uh oh!

NickM-27 Sep 12, 2024 Collaborator Sponsor

Uh oh!

Uh oh!

mahchet Sep 13, 2024 Author

Uh oh!

mahchet Sep 16, 2024 Author

Uh oh!

NickM-27 Sep 16, 2024 Collaborator Sponsor

Uh oh!

Uh oh!

mahchet Sep 16, 2024 Author

Uh oh!

NickM-27 Sep 16, 2024 Collaborator Sponsor

Uh oh!

mahchet Sep 16, 2024 Author

mahchet
Sep 12, 2024

Replies: 2 comments 7 replies

NickM-27
Sep 12, 2024
Collaborator Sponsor

mahchet
Sep 13, 2024
Author

mahchet Sep 16, 2024
Author

NickM-27 Sep 16, 2024
Collaborator Sponsor

mahchet Sep 16, 2024
Author

NickM-27 Sep 16, 2024
Collaborator Sponsor

mahchet Sep 16, 2024
Author