Sanity Check on CPU Usage and Issue adding 8th Camera #9790

adirburke · 2024-02-11T08:19:25Z

adirburke
Feb 11, 2024

Hi All,
Computer Specs
i9 9900k
32 GB 3200 RAM
A4000 16GB

The computer is running Proxmox, and frigate is running as a docker container on a ubuntu VM (only one VM running atm)
VM has 14 out of the 16 cores and 28GB RAM

compose.yml

version: "3.9"
services:
  frigate:
      container_name: frigate
      restart: unless-stopped
      image: ghcr.io/blakeblackshear/frigate:stable-tensorrt
      shm_size: 8G # update for your cameras based on calculation above
      volumes:
        - /etc/localtime:/etc/localtime:ro
        - ./config:/config
        - /home/adir/media/files/nvr:/media/frigate
        - type: tmpfs # Optional: 2GB of memory, reduces SSD/SD Card wear
          target: /tmp/cache
          tmpfs:
            size: 2000000000
      ports:
          - "5000:5000"
          - "8554:8554" # RTSP feeds
          - "8555:8555/tcp" # WebRTC over tcp
          - "8555:8555/udp" # WebRTC over udp
      environment:
        FRIGATE_RTSP_PASSWORD: "password"  #modify to whatever if using rtsp
      deploy:
        resources:
          reservations:
            devices:
              - driver: nvidia
                count: 1
                capabilities: [gpu]

config.yml

mqtt:
  enabled: False
  
detectors:
  tensorrt:
    type: tensorrt

model:
  path: /config/model_cache/tensorrt/yolov7-320.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 320
  height: 320

ffmpeg:
  hwaccel_args: preset-nvidia-h265

snapshots: 
  enabled: true

record:
  enabled: true
  retain:
    days: 3
    mode: all
  events:
    retain:
      default: 30
      mode: motion

detect:
  fps: 5
  # width: 1280
  # height: 720
  enabled: true

cameras:
  back_yard: 
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/unicast/c2/s0/live
          roles:
            - record
        - path: rtsp://admin:[email protected]:554/unicast/c2/s1/live
          roles:
            - detect
            - rtmp

  front_door:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/unicast/c6/s0/live
          roles:
            - record
        - path: rtsp://admin:[email protected]:554/unicast/c6/s1/live
          roles:
            - detect
            - rtmp
  front_stairs:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/unicast/c4/s0/live
          roles:
            - record
        - path: rtsp://admin:[email protected]:554/unicast/c4/s1/live
          roles:
            - detect
            - rtmp
  garage:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/unicast/c5/s0/live
          roles:
            - record
        - path: rtsp://admin:[email protected]:554/unicast/c5/s1/live
          roles:
            - detect
            - rtmp
  side_up:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/unicast/c1/s0/live
          roles:
            - record
        - path: rtsp://admin:[email protected]:554/unicast/c1/s1/live
          roles:
            - detect
            - rtmp
  side_down:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://admin:[email protected]:554/unicast/c8/s0/live
          roles:
            - record
        - path: rtsp://admin:[email protected]:554/unicast/c8/s1/live
          roles:
            - detect
            - rtmp
  front_gate_intercom:
    enabled: true
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
        - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1
          roles:
            - record
        - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0
          roles:
            - detect
            - rtmp
  # front_door_intercom:
  #   enabled: true
  #   ffmpeg:
  #     hwaccel_args: preset-nvidia-h264
  #     inputs:
  #       - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif
  #         roles:
  #           - record
  #       - path: rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
  #         roles:
  #           - detect
  #           - rtmp

When ever I add front_door_intercom in the config Frigate doesnt boot, it says detector is stalled and reboots

My Current CPU usage which this set is 50% (running 7 cameras)

Cameras:

Six are running these settings

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"20/1","codec_long_name":"H.265/HEVC(HighEfficiencyVideoCoding)","height":2160,"width":3840}]}},{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"20/1","codec_long_name":"H.265/HEVC(HighEfficiencyVideoCoding)","height":576,"width":704}]}}]

while the other two are running

[{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"25/1","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":1080,"width":1920},{"avg_frame_rate":"0/0","bit_rate":"256000","codec_long_name":"PCMsigned16-bitbig-endian"}]}},{"return_code":0,"stderr":"","stdout":{"programs":[],"streams":[{"avg_frame_rate":"25/1","codec_long_name":"H.264/AVC/MPEG-4AVC/MPEG-4part10","height":720,"width":1280},{"avg_frame_rate":"0/0","bit_rate":"256000","codec_long_name":"PCMsigned16-bitbig-endian"}]}}]

Frigate LOGS

2024-02-11 08:41:13.718966018  [INFO] Preparing Frigate...
2024-02-11 08:41:13.743425369  [INFO] Starting Frigate...
2024-02-11 08:41:15.420819489  [2024-02-11 08:41:15] frigate.app                    INFO    : Starting Frigate (0.13.1-34fb1c2)
2024-02-11 08:41:15.765691989  [hevc @ 0x399f380] PPS id out of range: 0
2024-02-11 08:41:15.766331678  [hevc @ 0x399f380] PPS id out of range: 0
2024-02-11 08:41:20.169408995  [hevc @ 0x399ed40] PPS id out of range: 0
2024-02-11 08:41:20.169411470  [hevc @ 0x399ed40] PPS id out of range: 0
2024-02-11 08:41:39.483610632  [2024-02-11 08:41:39] peewee_migrate.logs            INFO    : Starting migrations
2024-02-11 08:41:39.491862850  [2024-02-11 08:41:39] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-02-11 08:41:39.529419842  [2024-02-11 08:41:39] frigate.app                    INFO    : Recording process started: 758
2024-02-11 08:41:39.531999059  [2024-02-11 08:41:39] frigate.app                    INFO    : go2rtc process pid: 96
2024-02-11 08:41:39.556335774  [2024-02-11 08:41:39] frigate.app                    INFO    : Output process started: 769
2024-02-11 08:41:39.580550478  [2024-02-11 08:41:39] detector.tensorrt              INFO    : Starting detection process: 767
2024-02-11 08:41:39.732591499  [2024-02-11 08:41:39] frigate.app                    INFO    : Camera processor started for back_yard: 778
2024-02-11 08:41:39.732596616  [2024-02-11 08:41:39] frigate.app                    INFO    : Camera processor started for front_door: 779
2024-02-11 08:41:39.732602267  [2024-02-11 08:41:39] frigate.app                    INFO    : Camera processor started for front_stairs: 782
2024-02-11 08:41:39.741768121  [2024-02-11 08:41:39] frigate.app                    INFO    : Camera processor started for garage: 785
2024-02-11 08:41:39.741772225  [2024-02-11 08:41:39] frigate.app                    INFO    : Camera processor started for side_up: 787
2024-02-11 08:41:39.741773911  [2024-02-11 08:41:39] frigate.app                    INFO    : Camera processor started for side_down: 789
2024-02-11 08:41:39.741775516  [2024-02-11 08:41:39] frigate.app                    INFO    : Camera processor started for front_gate_intercom: 792
2024-02-11 08:41:39.741795271  [2024-02-11 08:41:39] frigate.app                    INFO    : Capture process started for back_yard: 796
2024-02-11 08:41:39.741797732  [2024-02-11 08:41:39] frigate.app                    INFO    : Capture process started for front_door: 799
2024-02-11 08:41:39.741799450  [2024-02-11 08:41:39] frigate.app                    INFO    : Capture process started for front_stairs: 821
2024-02-11 08:41:39.741818992  [2024-02-11 08:41:39] frigate.app                    INFO    : Capture process started for garage: 827
2024-02-11 08:41:39.741820659  [2024-02-11 08:41:39] frigate.app                    INFO    : Capture process started for side_up: 836
2024-02-11 08:41:39.741821965  [2024-02-11 08:41:39] frigate.app                    INFO    : Capture process started for side_down: 842
2024-02-11 08:41:39.741835356  [2024-02-11 08:41:39] frigate.app                    INFO    : Capture process started for front_gate_intercom: 849
2024-02-11 08:41:39.836528374  [2024-02-11 08:41:39] frigate.detectors.plugins.tensorrt INFO    : Loaded engine size: 72 MiB
2024-02-11 08:41:40.202083503  [2024-02-11 08:41:40] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +6, GPU +8, now: CPU 205, GPU 248 (MiB)
2024-02-11 08:41:40.227961177  [2024-02-11 08:41:40] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] Init cuDNN: CPU +2, GPU +10, now: CPU 207, GPU 258 (MiB)
2024-02-11 08:41:40.262144531  [2024-02-11 08:41:40] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +70, now: CPU 0, GPU 70 (MiB)
2024-02-11 08:41:40.272544108  [2024-02-11 08:41:40] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 135, GPU 250 (MiB)
2024-02-11 08:41:40.304851036  [2024-02-11 08:41:40] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 135, GPU 258 (MiB)
2024-02-11 08:41:40.305357257  [2024-02-11 08:41:40] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +15, now: CPU 0, GPU 85 (MiB)

Can I get some advice on lowering the CPU usage and getting the 8th camera to load

adirburke · 2024-02-11T08:55:28Z

adirburke
Feb 11, 2024
Author

LOG WHEN I TRY LOAD ALL 8

frigate  | 2024-02-11 08:53:23.210800725  10.30.0.55 - - [11/Feb/2024:08:53:23 +0000] "GET /api/config HTTP/1.1" 502 559 "http://10.30.0.15:5000/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36" "-"
frigate  | 2024-02-11 08:53:30.255658835  [INFO] Starting go2rtc healthcheck service...
frigate  | 2024-02-11 08:53:35.278176070  2024/02/11 08:53:35 [error] 151#151: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET /api/version HTTP/1.1", upstream: "http://127.0.0.1:5001version", host: "127.0.0.1:5000"
frigate  | 2024-02-11 08:53:50.384693956  2024/02/11 08:53:50 [error] 152#152: *5 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET /api/version HTTP/1.1", upstream: "http://127.0.0.1:5001version", host: "127.0.0.1:5000"
frigate  | 2024-02-11 08:53:59.187175977  [2024-02-11 08:53:59] peewee_migrate.logs            INFO    : Starting migrations
frigate  | 2024-02-11 08:53:59.197476528  [2024-02-11 08:53:59] peewee_migrate.logs            INFO    : There is nothing to migrate
frigate  | 2024-02-11 08:53:59.203424021  [2024-02-11 08:53:59] frigate.app                    INFO    : Recording process started: 793
frigate  | 2024-02-11 08:53:59.206067522  [2024-02-11 08:53:59] frigate.app                    INFO    : go2rtc process pid: 98
frigate  | 2024-02-11 08:53:59.231779006  [2024-02-11 08:53:59] frigate.app                    INFO    : Output process started: 802
frigate  | 2024-02-11 08:53:59.254388795  [2024-02-11 08:53:59] detector.tensorrt              INFO    : Starting detection process: 801
frigate  | 2024-02-11 08:53:59.341813543  [2024-02-11 08:53:59] frigate.app                    INFO    : Camera processor started for back_yard: 814
frigate  | 2024-02-11 08:53:59.342167827  [2024-02-11 08:53:59] frigate.app                    INFO    : Camera processor started for front_door: 816
frigate  | 2024-02-11 08:53:59.348975585  [2024-02-11 08:53:59] frigate.app                    INFO    : Camera processor started for front_stairs: 818
frigate  | 2024-02-11 08:53:59.349130093  [2024-02-11 08:53:59] frigate.app                    INFO    : Camera processor started for garage: 821
frigate  | 2024-02-11 08:53:59.349330461  [2024-02-11 08:53:59] frigate.app                    INFO    : Camera processor started for side_up: 824
frigate  | 2024-02-11 08:53:59.349486086  [2024-02-11 08:53:59] frigate.app                    INFO    : Camera processor started for side_down: 827
frigate  | 2024-02-11 08:53:59.349616664  [2024-02-11 08:53:59] frigate.app                    INFO    : Camera processor started for front_gate_intercom: 829
frigate  | 2024-02-11 08:53:59.349638593  [2024-02-11 08:53:59] frigate.app                    INFO    : Camera processor started for front_door_intercom: 832
frigate  | 2024-02-11 08:53:59.349797974  [2024-02-11 08:53:59] frigate.app                    INFO    : Capture process started for back_yard: 836
frigate  | 2024-02-11 08:53:59.349800000  [2024-02-11 08:53:59] frigate.app                    INFO    : Capture process started for front_door: 841
frigate  | 2024-02-11 08:53:59.349859108  [2024-02-11 08:53:59] frigate.app                    INFO    : Capture process started for front_stairs: 861
frigate  | 2024-02-11 08:53:59.349958326  [2024-02-11 08:53:59] frigate.app                    INFO    : Capture process started for garage: 864
frigate  | 2024-02-11 08:53:59.355859490  [2024-02-11 08:53:59] frigate.app                    INFO    : Capture process started for side_up: 867
frigate  | 2024-02-11 08:53:59.368886607  [2024-02-11 08:53:59] frigate.app                    INFO    : Capture process started for side_down: 871
frigate  | 2024-02-11 08:53:59.368991698  [2024-02-11 08:53:59] frigate.app                    INFO    : Capture process started for front_gate_intercom: 874
frigate  | 2024-02-11 08:53:59.379015946  [2024-02-11 08:53:59] frigate.app                    INFO    : Capture process started for front_door_intercom: 885
frigate  | 2024-02-11 08:53:59.653821088  [2024-02-11 08:53:59] frigate.detectors.plugins.tensorrt INFO    : Loaded engine size: 72 MiB
frigate  | 2024-02-11 08:54:00.057935203  [2024-02-11 08:54:00] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +6, GPU +8, now: CPU 206, GPU 248 (MiB)
frigate  | 2024-02-11 08:54:00.072053785  [2024-02-11 08:54:00] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] Init cuDNN: CPU +2, GPU +10, now: CPU 208, GPU 258 (MiB)
frigate  | 2024-02-11 08:54:00.095185023  [2024-02-11 08:54:00] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +70, now: CPU 0, GPU 70 (MiB)
frigate  | 2024-02-11 08:54:00.108353449  [2024-02-11 08:54:00] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 136, GPU 250 (MiB)
frigate  | 2024-02-11 08:54:00.141446611  [2024-02-11 08:54:00] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 136, GPU 258 (MiB)
frigate  | 2024-02-11 08:54:00.146893360  [2024-02-11 08:54:00] frigate.detectors.plugins.tensorrt INFO    : [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +15, now: CPU 0, GPU 85 (MiB)
frigate  | 2024-02-11 08:54:20.892366458  [2024-02-11 08:54:20] frigate.watchdog               INFO    : Detection appears to have stopped. Exiting Frigate...
frigate  | s6-rc: info: service legacy-services: stopping
frigate  | s6-rc: info: service legacy-services successfully stopped
frigate  | s6-rc: info: service nginx: stopping
frigate  | s6-rc: info: service go2rtc-healthcheck: stopping
frigate  | s6-rc: info: service go2rtc-healthcheck successfully stopped
frigate  | 2024-02-11 08:54:21.444749935  [INFO] The go2rtc-healthcheck service exited with code 256 (by signal 15)
frigate  | s6-rc: info: service nginx successfully stopped
frigate  | s6-rc: info: service nginx-log: stopping
frigate  | s6-rc: info: service frigate: stopping
frigate  | s6-rc: info: service nginx-log successfully stopped
frigate  | 2024-02-11 08:54:21.661475065  [INFO] Service NGINX exited with code 0 (by signal 0)
frigate  | 2024-02-11 08:54:22.389505308  [2024-02-11 08:54:21] frigate.app                    INFO    : Stopping...
frigate  | 2024-02-11 08:54:22.699076307  [2024-02-11 08:54:22] frigate.storage                INFO    : Exiting storage maintainer...
frigate  | 2024-02-11 08:54:22.930607258  [2024-02-11 08:54:22] frigate.events.cleanup         INFO    : Exiting event cleanup...
frigate  | 2024-02-11 08:54:22.939962703  [2024-02-11 08:54:22] frigate.ptz.autotrack          INFO    : Exiting autotracker...
frigate  | 2024-02-11 08:54:23.144812095  [2024-02-11 08:54:22] frigate.watchdog               INFO    : Exiting watchdog...
frigate  | 2024-02-11 08:54:23.147918071  [2024-02-11 08:54:22] frigate.stats                  INFO    : Exiting stats emitter...
frigate  | 2024-02-11 08:54:23.148814508  [2024-02-11 08:54:22] frigate.record.cleanup         INFO    : Exiting recording cleanup...
frigate  | 2024-02-11 08:54:23.151009279  [2024-02-11 08:54:22] frigate.object_processing      INFO    : Exiting object processor...
frigate  | 2024-02-11 08:54:23.321728115  [2024-02-11 08:54:23] frigate.events.maintainer      INFO    : Exiting event processor...
frigate  | 2024-02-11 08:54:23.589392384  Traceback (most recent call last):
frigate  | 2024-02-11 08:54:23.589611796    File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
frigate  | 2024-02-11 08:54:23.618226915      return _run_code(code, main_globals, None,
frigate  | 2024-02-11 08:54:23.618359122    File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
frigate  | 2024-02-11 08:54:23.618383279      exec(code, run_globals)
frigate  | 2024-02-11 08:54:23.618653050    File "/opt/frigate/frigate/__main__.py", line 17, in <module>
frigate  | 2024-02-11 08:54:23.620895619      frigate_app.start()
frigate  | 2024-02-11 08:54:23.621163275    File "/opt/frigate/frigate/app.py", line 710, in start
frigate  | 2024-02-11 08:54:23.623415629      self.flask_app.run(host="127.0.0.1", port=5001, debug=False, threaded=True)
frigate  | 2024-02-11 08:54:23.623418356    File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 889, in run
frigate  | 2024-02-11 08:54:23.625647553      run_simple(t.cast(str, host), port, self, **options)
frigate  | 2024-02-11 08:54:23.625783043    File "/usr/local/lib/python3.9/dist-packages/werkzeug/serving.py", line 1109, in run_simple
frigate  | 2024-02-11 08:54:23.629360207      srv.serve_forever()
frigate  | 2024-02-11 08:54:23.629363756    File "/usr/local/lib/python3.9/dist-packages/werkzeug/serving.py", line 806, in serve_forever
frigate  | 2024-02-11 08:54:23.629806111      super().serve_forever(poll_interval=poll_interval)
frigate  | 2024-02-11 08:54:23.630121071    File "/usr/lib/python3.9/socketserver.py", line 232, in serve_forever
frigate  | 2024-02-11 08:54:23.634096594      ready = selector.select(poll_interval)
frigate  | 2024-02-11 08:54:23.634278813    File "/usr/lib/python3.9/selectors.py", line 416, in select
frigate  | 2024-02-11 08:54:23.634667253      fd_event_list = self._selector.poll(timeout)
frigate  | 2024-02-11 08:54:23.634709855    File "/opt/frigate/frigate/app.py", line 704, in receiveSignal
frigate  | 2024-02-11 08:54:23.635141942      self.stop()
frigate  | 2024-02-11 08:54:23.635143603    File "/opt/frigate/frigate/app.py", line 725, in stop
frigate  | 2024-02-11 08:54:23.635393795      connection_id = self.detection_queue.get(timeout=1)
frigate  | 2024-02-11 08:54:23.635781433    File "/usr/lib/python3.9/multiprocessing/queues.py", line 109, in get
frigate  | 2024-02-11 08:54:23.636890887      raise Empty
frigate  | 2024-02-11 08:54:23.638476445  _queue.Empty

0 replies

adirburke · 2024-02-20T07:23:18Z

adirburke
Feb 20, 2024
Author

HI ALL

IT WAS RUNNING A VM IN PROXMOX, I MOVED TO LXC AND I HAVE NO ISSUES

Message me if you need help with moving (found some very good resources)

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Sanity Check on CPU Usage and Issue adding 8th Camera #9790

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

Sanity Check on CPU Usage and Issue adding 8th Camera #9790

Uh oh!

Uh oh!

adirburke Feb 11, 2024

compose.yml

config.yml

Replies: 2 comments

Uh oh!

adirburke Feb 11, 2024 Author

Uh oh!

adirburke Feb 20, 2024 Author

adirburke
Feb 11, 2024

adirburke
Feb 11, 2024
Author

adirburke
Feb 20, 2024
Author