[Support]: Frigate server regularly crashing?

[ReticentRobot]

Describe the problem you are having

My Frigate server randomly crashes on a regular basis (a bit of an oxymoron but I hope you get what I mean). I'm not much of a Linux user, so I'm having trouble tracking down what exactly is happening. I can't tell if it's the OS (Ubuntu), Docker, or Frigate causing the crash. It locks up the whole PC and I'm unable to remote into it (it is headless - but plugging a screen/mouse/keyboard into does not help, there is no picture or response at all when it has crashed), the only way to get everything back up and running is to hold the power down in person and force a hard power down, then turn it back on and everything is fine again.

I added a daily cron to restart the system hoping that would help, and it seemed to a bit - but after thinking that would fix the issue and it running fine for a few days - I just got a crash in the middle of the day again today.

I'm hoping someone can help me track down what is going on before I go away for a 2 week vacation, don't really want to lose access to all of my cameras and have them all stop recording while I'm away!

I found some things online that said to check /var/crash - but when I try to open any of the files there then VS Code says there is an error and it can't open the file. I then found some people saying to check using journalctl -b -1 -e - but I don't really see anything wrong there? Not really 100% what I'm looking at/for though. This is the output I get before the last crash today occured:

May 08 18:05:01 Frigate CRON[742646]: pam_unix(cron:session): session closed for user root
May 08 18:10:00 Frigate systemd[1]: Starting sysstat-collect.service - system activity accounting tool...
May 08 18:10:00 Frigate systemd[1]: sysstat-collect.service: Deactivated successfully.
May 08 18:10:00 Frigate systemd[1]: Finished sysstat-collect.service - system activity accounting tool.
May 08 18:10:55 Frigate systemd[1]: Starting fwupd-refresh.service - Refresh fwupd metadata and update motd...
May 08 18:10:55 Frigate systemd[1]: fwupd-refresh.service: Deactivated successfully.
May 08 18:10:55 Frigate systemd[1]: Finished fwupd-refresh.service - Refresh fwupd metadata and update motd.
May 08 18:15:01 Frigate CRON[749227]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 08 18:15:01 Frigate CRON[749228]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
May 08 18:15:01 Frigate CRON[749227]: pam_unix(cron:session): session closed for user root
May 08 18:17:01 Frigate CRON[750558]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 08 18:17:01 Frigate CRON[750559]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
May 08 18:17:01 Frigate CRON[750558]: pam_unix(cron:session): session closed for user root
May 08 18:20:00 Frigate systemd[1]: Starting sysstat-collect.service - system activity accounting tool...
May 08 18:20:00 Frigate systemd[1]: sysstat-collect.service: Deactivated successfully.
May 08 18:20:00 Frigate systemd[1]: Finished sysstat-collect.service - system activity accounting tool.
May 08 18:25:01 Frigate CRON[755921]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 08 18:25:01 Frigate CRON[755922]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
May 08 18:25:01 Frigate CRON[755921]: pam_unix(cron:session): session closed for user root
May 08 18:26:41 Frigate canonical-livepatch.canonical-livepatchd[1328]: Client information is recent, not refreshing.
May 08 18:26:41 Frigate canonical-livepatch.canonical-livepatchd[1328]: /usr/share/doc/linux-image-6.11.0-25-generic/changelog.Debian.gz
May 08 18:30:00 Frigate systemd[1]: Starting sysstat-collect.service - system activity accounting tool...
May 08 18:30:00 Frigate systemd[1]: sysstat-collect.service: Deactivated successfully.
May 08 18:30:00 Frigate systemd[1]: Finished sysstat-collect.service - system activity accounting tool.
May 08 18:30:01 Frigate CRON[759261]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
May 08 18:30:01 Frigate CRON[759263]: (root) CMD ([ -x /etc/init.d/anacron ] && if [ ! -d /run/systemd/system ]; then /usr/sbin/invoke-rc.d anacron start >/dev/null; fi)
May 08 18:30:01 Frigate CRON[759261]: pam_unix(cron:session): session closed for user root

Also worth noting that I've rebuilt this server twice and it was doing this on both of those builds as well.

I have Google Coral m.2 in the system. I used to have a USB Coral plugged in too, but I removed that before this latest rebuild in case that was causing the issue.

I have Timeshift installed for backups and Visual Studio Code installed, plus Docker and Frigate. Everything else is just stock Ubuntu.

Should I rebuild again except just use Debian? I thought Ubuntu would be safe as a Debian derivative.

Thanks for any help on tracking down what is happening!!

Version

0.15.0-cea210d

What browser(s) are you using?

Firefox 138.0.1 (64-bit)

Frigate config file

mqtt:
  enabled: true
  host: 192.168.1.102
  user: mqtt-client
  password: 

tls:
  enabled: false

detectors:
#  coral_usb:
#    type: edgetpu
#    device: usb
  coral_pci:
    type: edgetpu
    device: pci

go2rtc:
  streams:
    frontyard_a:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
      - ffmpeg:frontyard_a#audio=opus
    frontyard_a_sub:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1#backchannel=0
    backyard_a:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
      - ffmpeg:backyard_b#audio=opus
    backyard_a_sub:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1#backchannel=0
    backyard_b:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
      - ffmpeg:backyard_b#audio=opus
    backyard_b_sub:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1#backchannel=0
    backyard_c:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
      - ffmpeg:backyard_c#audio=opus
    backyard_c_sub:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1#backchannel=0
    backyard_d:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
      - ffmpeg:backyard_c#audio=opus
    backyard_d_sub:
      - rtsp://admin:[email protected]:554/cam/realmonitor?channel=1&subtype=1#backchannel=0
    front_doorbell:
      - ffmpeg:http://192.168.1.30/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=frigate&password=qXPf3pAz#video=copy#audio=copy#audio=opus
      - rtsp://192.168.1.30/Preview_01_sub
    front_doorbell_sub:
      - ffmpeg:http://192.168.1.30/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=frigate&password=qXPf3pAz

cameras:
  frontyard_a:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/frontyard_a
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/frontyard_a_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/frontyard_a?audio
          input_args: preset-rtsp-restream
          roles:
            - audio
    detect:
      enabled: true
    record:
      enabled: true
      retain:
        days: 30
        mode: all
    snapshots:
      enabled: true
    motion:
      mask:
        - 0.336,0,0,0,0,0.32,0.225,0.225,0.249,0.33,0.298,0.444,0.295,0.512,0.541,0.537,0.626,0.425,0.552,0.32,0.599,0.165,0.519,0
        - 0.513,0,0.572,0.222,0.746,0.285,1,0.413,1,0
        - 0.806,0.475,0.894,0.466,1,0.515,1,1,0.962,1,0.933,0.865,0.952,0.812,0.747,0.557,0.71,0.499
      threshold: 35
      contour_area: 20
    zones:
      sidewalk:
        coordinates: 
          0.069,0.587,0.147,0.545,0.235,0.69,0.489,0.831,0.668,0.927,0.828,0.996,0.495,0.999,0.176,0.787
        inertia: 3
        loitering_time: 0
        objects:
          - person
          - bird
          - cat
          - dog
      driveway:
        coordinates: 0.224,0.337,0,0.437,0,0.653,0.443,0.347
        inertia: 3
        loitering_time: 0
      lawn:
        coordinates: 
          1,0.47,0.634,0.376,0.455,0.342,0.148,0.545,0.24,0.693,0.496,0.836,0.815,0.996,1,1
        inertia: 3
        loitering_time: 0
      street:
        coordinates: 0.569,0.219,0.467,0.334,1,0.529,1,0.358
        inertia: 3
        loitering_time: 0
    review:
      alerts:
        required_zones:
          - sidewalk
          - driveway
          - lawn
          - street
      detections:
        required_zones:
          - sidewalk
          - driveway
          - lawn
          - street
    objects:
      filters:
        car:
          mask: 
            0.869,0.177,0.664,0.115,0.528,0.274,0.132,0.263,0,0.341,0.754,0.418,1,0.539,1,0.226
        person:
          mask:
            - 0.044,0.306,0.049,0.355,0.069,0.356,0.064,0.305
            - 0.207,0.214,0.213,0.265,0.241,0.266,0.235,0.222
            - 0.997,0.361,0.604,0.238,0.568,0.269,0.102,0.276,0.002,0,0.996,0.005
  backyard_a:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/backyard_a
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/backyard_a_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/backyard_a?audio
          input_args: preset-rtsp-restream
          roles:
            - audio
    detect:
      enabled: true
    record:
      enabled: true
      retain:
        days: 30
        mode: all
    snapshots:
      enabled: true
    motion:
      mask:
        - 0.511,0.407,0,0.371,0,0.001,0.559,0,0.56,0.149,0.602,0.156,0.601,0.295,0.534,0.314
        - 0.654,0,0.645,0.598,0.613,0.897,0.634,1,1,1,1,0
    objects:
      filters:
        person:
          mask: 0.706,0.373,0.545,0.416,0.534,0.832,0.555,1,0.778,1
  backyard_b:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/backyard_b
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/backyard_b_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/backyard_b?audio
          input_args: preset-rtsp-restream
          roles:
            - audio
    detect:
      enabled: true
    record:
      enabled: true
      retain:
        days: 30
        mode: all
    snapshots:
      enabled: true
    motion:
      threshold: 50
      contour_area: 35
      mask:
        - 0.519,0.256,0.587,0.289,0.628,0.305,0.892,0.385,0.934,0.537,0.997,0.47,0.983,0.16,0.9,0.075,0.763,0.08,0.727,0.051,0.646,0.058,0.644,0,0.559,0,0.517,0.112
        - 0.002,0.284,0.191,0.152,0.257,0.12,0.293,0.149,0.374,0.113,0.439,0.145,0.529,0.136,0.512,0.085,0.479,0.063,0.455,0.002,0.003,0.004
    zones:
      Frisbee_Golf:
        coordinates: 0.985,0.174,0.986,0.004,0.219,0.001,0.224,0.128,0.578,0.132
        loitering_time: 0
        objects:
          - person
          - dog
          - cat
          - car
    objects:
      filters:
        bird: {}
        car:
          mask:
            - 0.131,0.087,0.133,0.178,0.302,0.12,0.302,0.024
            - 0.546,-0.004,0.546,0.051,0.806,0.058,0.821,0.011
  backyard_c:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/backyard_c
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/backyard_c_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/backyard_c?audio
          input_args: preset-rtsp-restream
          roles:
            - audio
    detect:
      enabled: true
    record:
      enabled: true
      retain:
        days: 30
        mode: all
    snapshots:
      enabled: true
    motion:
      mask: 
        0.538,0.54,0.686,0.537,0.689,0.217,0.748,0.147,1,0.273,1,0,0,0,0,0.104,0.544,0.116
  backyard_d:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/backyard_d
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/backyard_d_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/backyard_d?audio
          input_args: preset-rtsp-restream
          roles:
            - audio
    detect:
      enabled: true
    record:
      enabled: true
      retain:
        days: 30
        mode: all
    snapshots:
      enabled: true
    motion:
      mask:
        - 0.108,0.784,0.169,0.74,0.239,0.722,0.283,0.762,0.288,0.814,0.189,0.878,0.13,0.862
        - 0.017,0.296,0.061,0.284,0.063,0.228,0.12,0.243,0.12,0.204,0.132,0.155,0.294,0.153,0.317,0.179,0.324,0.22,0.452,0.202,0.451,0.158,0.481,0.128,0.551,0.115,0.592,0.157,0.593,0.182,0.802,0.191,0.784,0.17,0.745,0.163,0.719,0.134,0.673,0.17,0.635,0.167,0.638,0.002,0.095,0,0.055,0.122,0,0.109,0.001,0.17,0.019,0.178
    objects:
      filters:
        person:
          mask: 0.321,0.375,0.636,0.413,0.637,0.191,0.633,-0.001,0.357,0,0.243,0
  front_doorbell:
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-aac
      inputs:
        - path: rtsp://127.0.0.1:8554/front_doorbell
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/front_doorbell_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/front_doorbell_sub?audio
          input_args: preset-rtsp-restream
          roles:
            - audio
      hwaccel_args: preset-vaapi
    detect:
      enabled: true
    record:
      enabled: true
      retain:
        days: 30
        mode: all
    snapshots:
      enabled: true
    motion:
      mask:
        - 0.739,0,0.511,0.373,0.506,0.612,0.458,0.714,0.409,1,0.867,1,1,1,1,0.58,1,0,0.916,0
        - 0.539,0.458,0.544,0.496,0.368,0.49,0.374,0.455
      threshold: 30
      contour_area: 10
    zones:
      zone_0:
        coordinates: 0,1,0,0.227,0.376,0.333,0.51,0.378,0.518,0.588,0.597,1
        inertia: 3
    review:
      alerts:
        required_zones: zone_0
      detections:
        required_zones: zone_0
    objects:
      filters:
        person:
          max_area: 400000
        car:
          mask: 
            0.394,0.421,0.374,0.438,0.376,0.496,0.499,0.506,0.542,0.514,0.634,0.523,0.582,0.47,0.513,0.414

detect:
  stationary:
    interval: 50
    threshold: 50

objects:
  track:
    - bird
    - car
    - cat
    - dog
    - package
    - person
version: 0.15-1

Relevant Frigate log output

Logs only seem to go back to the most recent boot

Relevant go2rtc log output

Logs only seem to go back to the most recent boot

FFprobe output from your camera

Not sure how to do this within the container?

Frigate stats

{"cameras":{"frontyard_a":{"camera_fps":4.8,"process_fps":4.4,"skipped_fps":0.5,"detection_fps":12.1,"detection_enabled":true,"pid":677,"capture_pid":725,"ffmpeg_pid":13651,"audio_rms":0.0,"audio_dBFS":0.0},"backyard_a":{"camera_fps":0.0,"process_fps":0.2,"skipped_fps":0.0,"detection_fps":0.0,"detection_enabled":true,"pid":678,"capture_pid":728,"ffmpeg_pid":20101,"audio_rms":0.0,"audio_dBFS":0.0},"backyard_b":{"camera_fps":0.0,"process_fps":0.2,"skipped_fps":0.0,"detection_fps":0.0,"detection_enabled":true,"pid":686,"capture_pid":732,"ffmpeg_pid":20098,"audio_rms":0.0,"audio_dBFS":0.0},"backyard_c":{"camera_fps":5.8,"process_fps":3.7,"skipped_fps":2.1,"detection_fps":15.7,"detection_enabled":true,"pid":697,"capture_pid":746,"ffmpeg_pid":19604,"audio_rms":0.0,"audio_dBFS":0.0},"backyard_d":{"camera_fps":0.0,"process_fps":0.0,"skipped_fps":0.0,"detection_fps":0.0,"detection_enabled":true,"pid":704,"capture_pid":749,"ffmpeg_pid":20044,"audio_rms":0.0,"audio_dBFS":0.0},"front_doorbell":{"camera_fps":6.7,"process_fps":3.2,"skipped_fps":3.5,"detection_fps":4.2,"detection_enabled":true,"pid":712,"capture_pid":753,"ffmpeg_pid":3510,"audio_rms":0.0,"audio_dBFS":0.0}},"detectors":{"coral_pci":{"inference_speed":7.63,"detection_start":0.0,"pid":585}},"detection_fps":32.0,"gpu_usages":{"intel-vaapi":{"gpu":"0.9%","mem":"-%","clients":{"3510":"7.79%","13651":"0.0%","19604":"0.0%"}}},"cpu_usages":{"frigate.full_system":{"cpu":"14.9","mem":"11.9"},"1":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/package/admin/s6/command/s6-svscan -d4 -- /run/service"},"15":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise s6-linux-init-shutdownd"},"17":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B"},"24":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise s6rc-oneshot-runner"},"25":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise frigate"},"26":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise go2rtc-log"},"27":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise nginx-log"},"28":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise certsync-log"},"29":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise go2rtc"},"30":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise s6rc-fdholder"},"31":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise nginx"},"32":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise go2rtc-healthcheck"},"33":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise frigate-log"},"34":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-supervise certsync"},"45":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules"},"46":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --"},"85":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx"},"88":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc"},"89":{"cpu":"0.1","cpu_average":"0","mem":"0.0","cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate"},"90":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/certsync"},"98":{"cpu":"4.0","cpu_average":"6","mem":"0.1","cmdline":"/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml"},"106":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"bash ./run.user go2rtc-healthcheck"},"107":{"cpu":"2.6","cpu_average":"3","mem":"1.5","cmdline":"python3 -u -m frigate"},"122":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: master process nginx"},"171":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"bash ./run.user certsync"},"189":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: worker process"},"190":{"cpu":"0.1","cpu_average":"0","mem":"0.0","cmdline":"nginx: worker process"},"191":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: worker process"},"192":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: worker process"},"198":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"nginx: cache manager process"},"323":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"sleep 9999"},"569":{"cpu":"1.4","cpu_average":"1","mem":"0.5","cmdline":"frigate.recording_manager"},"571":{"cpu":"0.2","cpu_average":"0","mem":"0.4","cmdline":"frigate.review_segment_manager"},"585":{"cpu":"7.0","cpu_average":"3","mem":"0.4","cmdline":"frigate.detector.coral_pci"},"604":{"cpu":"3.1","cpu_average":"2","mem":"0.4","cmdline":"frigate.output       "},"619":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"626":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"633":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"640":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"647":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"654":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"661":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -threads 1 -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -threads 1 -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:"},"677":{"cpu":"44.3","cpu_average":"12","mem":"0.5","cmdline":"frigate.process:frontyard_a"},"678":{"cpu":"0.0","cpu_average":"11","mem":"0.4","cmdline":"frigate.process:backyard_a"},"686":{"cpu":"0.0","cpu_average":"5","mem":"0.5","cmdline":"frigate.process:backyard_b"},"697":{"cpu":"30.3","cpu_average":"13","mem":"0.5","cmdline":"frigate.process:backyard_c"},"704":{"cpu":"0.0","cpu_average":"0","mem":"0.4","cmdline":"frigate.process:backyard_d"},"712":{"cpu":"13.8","cpu_average":"3","mem":"0.4","cmdline":"frigate.process:front_doorbell"},"725":{"cpu":"1.8","cpu_average":"1","mem":"0.4","cmdline":"frigate.capture:frontyard_a"},"728":{"cpu":"0.3","cpu_average":"0","mem":"0.4","cmdline":"frigate.capture:backyard_a"},"732":{"cpu":"0.2","cpu_average":"0","mem":"0.4","cmdline":"frigate.capture:backyard_b"},"746":{"cpu":"1.1","cpu_average":"1","mem":"0.4","cmdline":"frigate.capture:backyard_c"},"749":{"cpu":"0.3","cpu_average":"0","mem":"0.4","cmdline":"frigate.capture:backyard_d"},"753":{"cpu":"1.8","cpu_average":"1","mem":"0.5","cmdline":"frigate.capture:front_doorbell"},"3510":{"cpu":"4.6","cpu_average":"3","mem":"0.2","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/front_doorbell_sub -r 5 -vf fps=5,scale_vaapi=w=1280:h=720,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"3536":{"cpu":"0.6","cpu_average":"0","mem":"0.1","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -v error -fflags nobuffer -flags low_delay -i http://192.168.1.30/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=*&password=* -c copy -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/a0169e894a3b8aa8755f37257320ca2c"},"13651":{"cpu":"5.2","cpu_average":"3","mem":"0.1","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/frontyard_a_sub -r 5 -vf fps=5,scale_vaapi=w=1280:h=720,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"14217":{"cpu":"4.6","cpu_average":"3","mem":"0.1","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -v error -fflags nobuffer -flags low_delay -i http://192.168.1.30/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=*&password=* -map 0:v:0? -c:v:0 copy -map 0:a:0? -c:a:0 copy -map 0:a:0? -c:a:1 libopus -application:a:1 lowdelay -min_comp 0 -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp rtsp://127.0.0.1:8554/af0994a6efb52a6f373e78c89ed74c12"},"14592":{"cpu":"0.8","cpu_average":"5","mem":"0.1","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/front_doorbell -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/front_doorbell@%Y%m%d%H%M%S%z.mp4"},"19604":{"cpu":"2.8","cpu_average":"0","mem":"0.1","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/backyard_c_sub -r 5 -vf fps=5,scale_vaapi=w=1280:h=720,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"19606":{"cpu":"2.0","cpu_average":"0","mem":"0.1","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/backyard_c -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/backyard_c@%Y%m%d%H%M%S%z.mp4"},"19876":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"sleep 30s"},"19954":{"cpu":"2.9","cpu_average":"0","mem":"0.1","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/frontyard_a -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/frontyard_a@%Y%m%d%H%M%S%z.mp4"},"20098":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/backyard_b_sub -r 5 -vf fps=5,scale_vaapi=w=1280:h=720,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"20100":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/backyard_b -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/backyard_b@%Y%m%d%H%M%S%z.mp4"},"20101":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -hwaccel_flags allow_profile_mismatch -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/backyard_a_sub -r 5 -vf fps=5,scale_vaapi=w=1280:h=720,hwdownload,format=nv12,eq=gamma=1.4:gamma_weight=0.5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:"},"20103":{"cpu":"0.0","cpu_average":"0","mem":"0.0","cmdline":"/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -loglevel warning -threads 2 -user_agent FFmpeg Frigate/0.15.0-cea210d -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/backyard_a -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/backyard_a@%Y%m%d%H%M%S%z.mp4"}},"service":{"uptime":1976,"version":"0.15.0-cea210d","latest_version":"0.15.1","storage":{"/media/frigate/recordings":{"total":7599959.2,"used":2613735.2,"free":4603133.8,"mount_type":"ext4"},"/media/frigate/clips":{"total":7599959.2,"used":2613735.2,"free":4603133.8,"mount_type":"ext4"},"/tmp/cache":{"total":953.7,"used":10.8,"free":942.9,"mount_type":"tmpfs"},"/dev/shm":{"total":512.0,"used":320.9,"free":191.1,"mount_type":"tmpfs"}},"temperatures":{"apex_0":44.3},"last_updated":1746817807},"processes":{"recording":{"pid":569},"review_segment":{"pid":571},"go2rtc":{"pid":98}}}

Install method

Docker Compose

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    restart: unless-stopped
    stop_grace_period: 30s
    image: ghcr.io/blakeblackshear/frigate:stable
    privileged: true
    shm_size: "512mb" # update for your cameras based on calculation above
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config:/config
      - /mnt/9d067f1b-3a7b-4a68-a868-c33ea3a190b0:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    devices:
      #- /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
    ports:
      - "5000:5000"
      - "8971:8971"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp

Object Detector

Coral

Network connection

Wired

Camera make and model

Amcrest ASH43-W

Screenshots of the Frigate UI's System metrics pages

system just crashed again while typing this up! Trying to get my wife to restart it again if she'll look at her phone and see my message. I'll update with screenshots here when I can.

Any other information that may be helpful

No response

[NickM-27]

you're going to need to capture syslogs, there is no way to know why this might be happening

[ReticentRobot]

It last crashed on the 16th, I left it down until this morning because today was the first day I had a little time to work on it. Looking through, I think maybe it is failing a firmware update and then crashing?
syslog.txt

[ReticentRobot]

If that's what is happening, everything looks up to date on the firmware except for this UEFI thing

I guess I will try to manually update that or see if I can figure out how to turn off it trying to auto update that and see if that stops the crashes.

[ReticentRobot]

Well I guess that wasn't it, had another full system crash/lockup.
syslog.txt

These are the last few things that happened right before the last lockup:

025-05-19T10:41:38.006543-05:00 Frigate containerd[1361]: time="2025-05-19T10:41:38.006439455-05:00" level=info msg="loading plugin "io.containerd.nri.v1.nri"..." type=io.containerd.nri.v1
2025-05-19T10:41:38.006565-05:00 Frigate containerd[1361]: time="2025-05-19T10:41:38.006457192-05:00" level=info msg="NRI interface is disabled by configuration."
2025-05-19T10:41:38.006814-05:00 Frigate containerd[1361]: time="2025-05-19T10:41:38.006787339-05:00" level=info msg=serving... address=/run/containerd/containerd.sock.ttrpc
2025-05-19T10:41:38.006851-05:00 Frigate containerd[1361]: time="2025-05-19T10:41:38.006823005-05:00" level=info msg=serving... address=/run/containerd/containerd.sock
2025-05-19T10:41:38.006906-05:00 Frigate containerd[1361]: time="2025-05-19T10:41:38.006885370-05:00" level=info msg="containerd successfully booted in 0.039857s"
2025-05-19T10:41:38.006959-05:00 Frigate systemd[1]: Started containerd.service - containerd container runtime.
2025-05-19T10:41:38.016581-05:00 Frigate systemd[1]: Finished [email protected] - User Runtime Directory /run/user/120.
2025-05-19T10:41:38.018341-05:00 Frigate systemd[1]: Starting [email protected] - User Manager for UID 120...
2025-05-19T10:41:38.045232-05:00 Frigate systemd[1]: Started cups.service - CUPS Scheduler.
2025-05-19T10:41:38.151866-05:00 Frigate systemd[1]: tmp-snap.rootfs_J6ywNp.mount: Deactivated successfully.
2025-05-19T10:41:38.162897-05:00 Frigate systemd[1]: rsyslog.service: Sent signal SIGHUP to main process 1000 (rsyslogd) on client request.

[ReticentRobot]

I found someone say to use apport-retrace to read the files in /var/crash, I had these files in there:

_usr_bin_ibus-daemon.120.crash

E: You must put some 'deb-src' URIs in your sources.list
/usr/lib/python3/dist-packages/apport/packaging_impl/apt_dpkg.py:775: Warning: W:Download is performed unsandboxed as root as file '/tmp/tmp1va1l2lm/ibus_1.5.29-2.debian.tar.xz' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
result = fetcher.run()
dpkg-source: warning: -sn is not a valid option for Dpkg::Source::Package::V3::Quilt
gpgv: Signature made Sun 31 Mar 2024 07:11:31 AM CDT
gpgv: using RSA key 7E7729476D87D6F11D91ACCBC293E7B461825ACE
gpgv: Can't check signature: No public key
dpkg-source: warning: cannot verify inline signature for /tmp/tmp1va1l2lm/ibus_1.5.29-2.dsc: no acceptable signature found
Traceback (most recent call last):
File "/usr/bin/apport-retrace", line 696, in
main()
File "/usr/bin/apport-retrace", line 691, in main
with open(options.output or options.report, "wb") as out:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/var/crash/_usr_bin_ibus-daemon.120.crash'

_usr_share_ibus-table_engine_main.py.120.crash
ERROR: report file does not contain one of the required fields: CoreDump

Not sure if any of that information is useful (will see what I can find out) - but thought I would go ahead and paste it here. I think the top one is saying that something went wrong with apport-retrace, and not what's actually in the file though?

[rangerdude]

Just FYI, I am having the same exact issue.
Details:

Host - Truenas
Container - Docker
AMD CPU and AMD GPU

I think I have tracked it down to a section of memory that reports out of bounds. Looks like buffer over-run. The memory before is kernel and just after is system IO.

Need a low level person to look at what this is doing, might need OS patch.

[ReticentRobot]

In my case I had two problems going on. First, my CMOS battery was dead and second, I had an old Nvidia GPU (730 I think?) that was throwing errors left and right. After replacing the CMOS battery and removing that card I no longer have any issues with crashes or it not recovering from power loss/reboot situations on some occasions.