[HW Accel Support]: RK3566 ffmpeg h264_rkmpp_decoder not working #9952
-
Describe the problem you are havingI compiled Armbian with 5.10 kernel for HA Green and started hardware decoding, decoding works fine in this project jellyfin-mpp but it doesn't work on Frigate and the CPU usage is high Version0.13.2-6476F8A Frigate config filemqtt:
enabled: false
ffmpeg:
hwaccel_args: preset-rk-h264
cameras:
living_camera:
ffmpeg:
inputs:
- path: rtsp://admin:[email protected]:554/h264/ch1/main/av_stream
roles:
- detect
detectors:
rknn:
type: rknn
core_mask: 0
model:
path: /config/model_cache/rknn/yolov8n-320x320-rk3566.rknn
width: 320
height: 320
input_pixel_format: bgr
input_tensor: nhwcdocker-compose file or Docker CLI commandversion: "3"
services:
frigate:
image: ghcr.io/blakeblackshear/frigate:stable-rk
container_name: frigate
volumes:
- /etc/localtime:/etc/localtime:ro
- /opt/frigate/conf:/config
- /opt/frigate/data:/media/frigate
- type: tmpfs
target: /tmp/cache
tmpfs:
size: 1gb
environment:
- TZ=Asia/Shanghai
- FRIGATE_RTSP_PASSWORD=frigate
network_mode: host
restart: always
privileged: true
shm_size: 256mbRelevant log output2024-02-21 11:42:26.481646742 [INFO] Preparing Frigate...
2024-02-21 11:42:26.493911398 [INFO] Starting NGINX...
2024-02-21 11:42:26.520921059 [INFO] Preparing new go2rtc config...
s6-rc: info: service legacy-services successfully started
2024-02-21 11:42:26.610056965 [INFO] Starting Frigate...
2024-02-21 11:42:29.578649515 [INFO] Starting go2rtc...
2024-02-21 11:42:29.960838082 11:42:29.960 INF go2rtc version 1.8.4 linux/arm64
2024-02-21 11:42:29.964977149 11:42:29.964 INF [rtsp] listen addr=:8554
2024-02-21 11:42:29.965002816 11:42:29.964 INF [api] listen addr=:1984
2024-02-21 11:42:29.966860743 11:42:29.966 INF [webrtc] listen addr=:8555
2024-02-21 11:42:30.990325462 2024/02/21 11:42:30 [error] 147#147: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET /api/version HTTP/1.1", upstream: "http://127.0.0.1:5001version", host: "127.0.0.1:5000"
2024-02-21 11:42:36.241627521 2024/02/21 11:42:36 [error] 148#148: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET /api/version HTTP/1.1", upstream: "http://127.0.0.1:5001version", host: "127.0.0.1:5000"
2024-02-21 11:42:36.464932351 [INFO] Starting go2rtc healthcheck service...
2024-02-21 11:42:41.427707858 [2024-02-21 11:42:41] frigate.app INFO : Starting Frigate (0.13.2-6476f8a)
2024-02-21 11:42:41.497234897 2024/02/21 11:42:41 [error] 149#149: *5 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET /api/version HTTP/1.1", upstream: "http://127.0.0.1:5001version", host: "127.0.0.1:5000"
2024-02-21 11:42:43.887294128 [2024-02-21 11:42:43] peewee_migrate.logs INFO : Starting migrations
2024-02-21 11:42:43.932186270 [2024-02-21 11:42:43] peewee_migrate.logs INFO : There is nothing to migrate
2024-02-21 11:42:43.966195681 [2024-02-21 11:42:43] frigate.app INFO : Recording process started: 343
2024-02-21 11:42:44.001428057 [2024-02-21 11:42:43] frigate.app INFO : go2rtc process pid: 90
2024-02-21 11:42:44.147228717 [2024-02-21 11:42:44] detector.rknn INFO : Starting detection process: 352
2024-02-21 11:42:44.160523838 [2024-02-21 11:42:44] frigate.app INFO : Output process started: 354
2024-02-21 11:42:44.254805649 [2024-02-21 11:42:44] frigate.app INFO : Camera processor started for living_camera: 359
2024-02-21 11:42:44.303447189 [2024-02-21 11:42:44] frigate.app INFO : Capture process started for living_camera: 361
2024-02-21 11:42:45.608617794 I RKNN: [11:42:45.608] RKNN Runtime Information: librknnrt version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
2024-02-21 11:42:45.608904795 I RKNN: [11:42:45.608] RKNN Driver Information: version: 0.9.2
2024-02-21 11:42:45.612056272 I RKNN: [11:42:45.611] RKNN Model Information: version: 6, toolkit version: 1.5.2+b642f30c(compiler version: 1.5.2 (c6b7b351a@2023-08-23T07:39:01)), target: RKNPU lite, target platform: rk3566, framework name: ONNX, framework layout: NCHW, model inference type: static_shape
2024-02-21 11:43:43.450434985 192.168.2.14 - - [21/Feb/2024:11:43:43 +0800] "GET / HTTP/1.1" 200 6636 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:43:43.650792803 192.168.2.14 - - [21/Feb/2024:11:43:43 +0800] "GET /api/config HTTP/1.1" 200 7703 "http://192.168.2.200:5000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:43:43.766321783 192.168.2.14 - - [21/Feb/2024:11:43:43 +0800] "GET /api/living_camera/latest.jpg?h=100 HTTP/1.1" 499 0 "http://192.168.2.200:5000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:43:44.060089032 192.168.2.14 - - [21/Feb/2024:11:43:44 +0800] "GET /api/living_camera/latest.jpg?h=296 HTTP/1.1" 200 6224 "http://192.168.2.200:5000/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:43:48.827891212 192.168.2.14 - - [21/Feb/2024:11:43:48 +0800] "GET /api/go2rtc/api HTTP/1.1" 200 143 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:43:48.941794723 192.168.2.14 - - [21/Feb/2024:11:43:48 +0800] "GET /api/config HTTP/1.1" 200 7703 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:43:59.415928505 192.168.2.14 - - [21/Feb/2024:11:43:59 +0800] "GET /system HTTP/1.1" 200 6636 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:43:59.430831594 192.168.2.14 - - [21/Feb/2024:11:43:59 +0800] "GET /ws HTTP/1.1" 101 6379 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:43:59.564599182 192.168.2.14 - - [21/Feb/2024:11:43:59 +0800] "GET /api/config HTTP/1.1" 200 7703 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:43:59.595377782 192.168.2.14 - - [21/Feb/2024:11:43:59 +0800] "GET /api/go2rtc/api HTTP/1.1" 200 143 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:46:11.213174834 [2024-02-21 11:46:11] frigate.video ERROR : living_camera: Unable to read frames from ffmpeg process.
2024-02-21 11:46:11.216738440 [2024-02-21 11:46:11] frigate.video ERROR : living_camera: ffmpeg process is not running. exiting capture thread...
2024-02-21 11:46:14.502915473 [2024-02-21 11:46:14] watchdog.living_camera ERROR : Ffmpeg process crashed unexpectedly for living_camera.
2024-02-21 11:46:14.507175876 [2024-02-21 11:46:14] watchdog.living_camera ERROR : The following ffmpeg logs include the last 100 lines prior to exit.
2024-02-21 11:46:14.508339634 [2024-02-21 11:46:14] ffmpeg.living_camera.detect ERROR : [rtsp @ 0xaaaaee805c70] CSeq 9 expected, 6 received.
2024-02-21 11:48:34.596499119 [2024-02-21 11:48:34] watchdog.living_camera INFO : No frames received from living_camera in 20 seconds. Exiting ffmpeg...
2024-02-21 11:48:34.598676133 [2024-02-21 11:48:34] watchdog.living_camera INFO : Waiting for ffmpeg to exit gracefully...
2024-02-21 11:48:47.369115733 [2024-02-21 11:48:47] frigate.video ERROR : living_camera: Unable to read frames from ffmpeg process.
2024-02-21 11:48:47.371982835 [2024-02-21 11:48:47] frigate.video ERROR : living_camera: ffmpeg process is not running. exiting capture thread...
2024-02-21 11:48:57.880544018 [2024-02-21 11:48:57] watchdog.living_camera ERROR : Ffmpeg process crashed unexpectedly for living_camera.
2024-02-21 11:48:57.881772234 [2024-02-21 11:48:57] watchdog.living_camera ERROR : The following ffmpeg logs include the last 100 lines prior to exit.
2024-02-21 11:48:57.883346078 [2024-02-21 11:48:57] ffmpeg.living_camera.detect ERROR : [rtsp @ 0xaaaae5dccc70] CSeq 7 expected, 6 received.
2024-02-21 11:48:57.884643711 [2024-02-21 11:48:57] ffmpeg.living_camera.detect ERROR : [out_0_0 @ 0xaaaae5e87c60] 100 buffers queued in out_0_0, something may be wrong.
2024-02-21 11:49:57.083114442 192.168.2.14 - - [21/Feb/2024:11:49:57 +0800] "GET /ws HTTP/1.1" 101 31633 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:50:23.283145953 [2024-02-21 11:50:23] frigate.video ERROR : living_camera: Unable to read frames from ffmpeg process.
2024-02-21 11:50:23.284973840 [2024-02-21 11:50:23] frigate.video ERROR : living_camera: ffmpeg process is not running. exiting capture thread...
2024-02-21 11:50:28.096202429 [2024-02-21 11:50:28] watchdog.living_camera ERROR : Ffmpeg process crashed unexpectedly for living_camera.
2024-02-21 11:50:28.097370854 [2024-02-21 11:50:28] watchdog.living_camera ERROR : The following ffmpeg logs include the last 100 lines prior to exit.
2024-02-21 11:50:28.098760946 [2024-02-21 11:50:28] ffmpeg.living_camera.detect ERROR : [rtsp @ 0xaaaaf2496c70] CSeq 8 expected, 6 received.
2024-02-21 11:50:28.099766328 [2024-02-21 11:50:28] ffmpeg.living_camera.detect ERROR : [rtsp @ 0xaaaaf2496c70] CSeq 11 expected, 7 received.
2024-02-21 11:51:57.396728522 192.168.2.14 - - [21/Feb/2024:11:51:57 +0800] "GET /ws HTTP/1.1" 101 6496 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:53:57.798878695 192.168.2.14 - - [21/Feb/2024:11:53:57 +0800] "GET /ws HTTP/1.1" 101 6379 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:56:30.391409386 192.168.2.14 - - [21/Feb/2024:11:56:30 +0800] "GET /api/go2rtc/api HTTP/1.1" 200 143 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:56:30.439038908 192.168.2.14 - - [21/Feb/2024:11:56:30 +0800] "GET /api/config HTTP/1.1" 200 7703 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:56:32.954513804 192.168.2.14 - - [21/Feb/2024:11:56:32 +0800] "GET /system HTTP/1.1" 200 6636 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:56:32.962468488 192.168.2.14 - - [21/Feb/2024:11:56:32 +0800] "GET /ws HTTP/1.1" 101 12914 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:56:33.077682056 192.168.2.14 - - [21/Feb/2024:11:56:33 +0800] "GET /api/config HTTP/1.1" 200 7703 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:56:33.132693425 192.168.2.14 - - [21/Feb/2024:11:56:33 +0800] "GET /api/go2rtc/api HTTP/1.1" 200 143 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:58:04.981493845 192.168.2.14 - - [21/Feb/2024:11:58:04 +0800] "GET /api/go2rtc/api HTTP/1.1" 200 143 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:58:05.066576563 192.168.2.14 - - [21/Feb/2024:11:58:05 +0800] "GET /api/config HTTP/1.1" 200 7703 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 11:58:58.521893281 192.168.2.14 - - [21/Feb/2024:11:58:58 +0800] "GET /ws HTTP/1.1" 101 12808 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 12:00:58.895241260 192.168.2.14 - - [21/Feb/2024:12:00:58 +0800] "GET /ws HTTP/1.1" 101 6433 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 12:02:59.185396039 192.168.2.14 - - [21/Feb/2024:12:02:59 +0800] "GET /ws HTTP/1.1" 101 6379 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 12:04:28.282608798 [2024-02-21 12:04:28] watchdog.living_camera INFO : No frames received from living_camera in 20 seconds. Exiting ffmpeg...
2024-02-21 12:04:28.286076465 [2024-02-21 12:04:28] watchdog.living_camera INFO : Waiting for ffmpeg to exit gracefully...
2024-02-21 12:04:58.315181417 [2024-02-21 12:04:58] watchdog.living_camera INFO : FFmpeg did not exit. Force killing...
2024-02-21 12:04:58.662129791 [2024-02-21 12:04:58] frigate.video ERROR : living_camera: Unable to read frames from ffmpeg process.
2024-02-21 12:04:58.665921503 [2024-02-21 12:04:58] frigate.video ERROR : living_camera: ffmpeg process is not running. exiting capture thread...
2024-02-21 12:04:59.511582991 192.168.2.14 - - [21/Feb/2024:12:04:59 +0800] "GET /ws HTTP/1.1" 101 6379 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 12:05:08.668619570 [2024-02-21 12:05:08] watchdog.living_camera ERROR : Ffmpeg process crashed unexpectedly for living_camera.
2024-02-21 12:05:08.669769334 [2024-02-21 12:05:08] watchdog.living_camera ERROR : The following ffmpeg logs include the last 100 lines prior to exit.
2024-02-21 12:05:08.671278144 [2024-02-21 12:05:08] ffmpeg.living_camera.detect ERROR : [rtsp @ 0xaaaae3036c70] CSeq 18 expected, 6 received.
2024-02-21 12:05:08.672704702 [2024-02-21 12:05:08] ffmpeg.living_camera.detect ERROR : [rtsp @ 0xaaaae3036c70] CSeq 27 expected, 8 received.
2024-02-21 12:05:08.674289346 [2024-02-21 12:05:08] ffmpeg.living_camera.detect ERROR : [out_0_0 @ 0xaaaae31b53f0] 100 buffers queued in out_0_0, something may be wrong.
2024-02-21 12:06:59.915204873 192.168.2.14 - - [21/Feb/2024:12:06:59 +0800] "GET /ws HTTP/1.1" 101 6432 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 12:10:00.445698443 192.168.2.14 - - [21/Feb/2024:12:10:00 +0800] "GET /ws HTTP/1.1" 101 12817 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 12:12:00.765456110 192.168.2.14 - - [21/Feb/2024:12:12:00 +0800] "GET /ws HTTP/1.1" 101 6476 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 12:13:42.423274358 192.168.2.14 - - [21/Feb/2024:12:13:42 +0800] "GET /api/go2rtc/api HTTP/1.1" 200 143 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 12:13:42.468440651 192.168.2.14 - - [21/Feb/2024:12:13:42 +0800] "GET /api/config HTTP/1.1" 200 7703 "http://192.168.2.200:5000/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"
2024-02-21 12:14:01.145335909 192.168.2.14 - - [21/Feb/2024:12:14:01 +0800] "GET /ws HTTP/1.1" 101 6529 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0" "-"FFprobe output from your cameraffprobe version e243e8d001-20231113 Copyright (c) 2007-2023 the FFmpeg developers
built with gcc 13.2.0 (crosstool-NG 1.25.0.232_c175b21)
configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=aarch64-ffbuild-linux-gnu- --arch=aarch64 --target-os=linux --enable-nonfree --enable-gpl --enable-version3 --disable-debug --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-openssl --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf --enable-libxcb --enable-xlib --enable-amf --enable-libaom --enable-libaribb24 --enable-avisynth --enable-chromaprint --enable-libdav1d --disable-libdavs2 --enable-libdrm --enable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar --enable-libass --enable-libbluray --disable-libjxl --enable-libmp3lame --enable-libopus --enable-librist --enable-libssh --enable-libtheora --disable-libvpx --enable-libwebp --enable-lv2 --enable-rkmpp --disable-libvpl --enable-openal --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-librav1e --enable-librubberband --disable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libdrm --disable-vaapi --enable-libvidstab --enable-vulkan --enable-libshaderc --disable-libplacebo --enable-libx264 --enable-libx265 --disable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags=-pie --extra-libs='-ldl -lstdc++ -lgomp' --extra-version=20231113
libavutil 58. 2.100 / 58. 2.100
libavcodec 60. 3.100 / 60. 3.100
libavformat 60. 3.100 / 60. 3.100
libavdevice 60. 1.100 / 60. 1.100
libavfilter 9. 3.100 / 9. 3.100
libswscale 7. 1.100 / 7. 1.100
libswresample 4. 10.100 / 4. 10.100
libpostproc 57. 1.100 / 57. 1.100
Input #0, rtsp, from 'rtsp://admin:[email protected]:554/h264/ch1/main/av_stream':
Metadata:
title : Media Presentation
Duration: N/A, start: 0.066644, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1920x1080, 14.99 fps, 14.99 tbr, 90k tbn
[h264_rkmpp_decoder @ 0xaaaaf76c2e60] Picture format is yuv420p.Operating systemOther Linux Install methodDocker Compose Network connectionWired Camera make and modelCS-C6TC-32WFR Any other information that may be helpful
|
Beta Was this translation helpful? Give feedback.
Answered by
nyanmisaka
Feb 23, 2024
Replies: 19 comments 25 replies
-
|
CC @MarcA711 |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
More frigate log dmesg log |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Could you try out if the new ffmpeg version works better? To do so: Afterwards change your docker compose to your local image by replacing the |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
Can you compile a test version and push it to dockerhub, I have a hard time compiling it in China due to GFW, I have tried many times and failed |
Beta Was this translation helpful? Give feedback.
-
|
sure, it is available under |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Debugging is OK, and the CPU usage is reduced a little. Similarly, decoding in Jellyfin will have a much lower CPU usage, but Jellyfin does not call NPU. I don't know if the NPU will affect the CPU usage or if there is a problem with my frigate configuration
There is a delay of more than a dozen seconds in the processing time. The Armbian I compiled was not configured for the CPU frequency, and it may be running at the lowest frequency, so I'll try again when the CPU frequency is fixed
|
Beta Was this translation helpful? Give feedback.
2 replies
-
Jellyfin uses a end-to-end hardware filter pipeline, which mainly includes scaling, tone-mapping and drawing subtitle overlays. Therefore, many CPU cycles are saved, and the CPU is mainly responsible for scheduling. If frigate wants to achieve this, you need to make the corresponding video filter in ffmpeg. For example, Intel's OpenVINO has filters such as vf_dnn_detect/classify in ffmpeg. They can directly use the HW texture produced by the decoder to avoid unnecessary memory copies, so that the HW encoder can also directly encode the post-processed image. |
Beta Was this translation helpful? Give feedback.
-
|
Thx i see |
Beta Was this translation helpful? Give feedback.
-
|
frigate log ffprobe log |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I tried lowering the detect resolution to reduce the CPU usage, but the screen is green Native resolution 1080p Is it a ffmpeg mpp related problem or a parameter problem?
Reducing the detect resolution seems to have an effect, but how to fix a green screen
|
Beta Was this translation helpful? Give feedback.
1 reply
-
Beta Was this translation helpful? Give feedback.
Answer selected by
liwei19920307
-
|
Thank you guys, high CPU usage and green screen issue solved @MarcA711 @nyanmisaka
Just replace the argument with this commit as @nyanmisaka says I added two cameras and didn't modify the resolution, one 1080p, one 480p, rkmmp decoding works fine, the CPU usage is very low, and even though I modified the resolution of the input, no green screen appeared @MarcA711 Can you fix this problem on the mainline, I achieved it by directly modifying the file fmpeg_presets.py in docker image, and currently the format of my camera is yuvj420p I don't know if the configuration of other models of cameras is valid This is my current configuration and I hope it will be useful to those who need it |
Beta Was this translation helpful? Give feedback.
6 replies
-
|
Please compile a new version and push it to dockerhub, I'll find more people to test it |
Beta Was this translation helpful? Give feedback.
-
|
Hey, sorry for the late reply. Currently, I try to get these presets working with afbc as I wrote here #9912 (comment). |
Beta Was this translation helpful? Give feedback.
-
|
I finally was able to get hardware acceleration working with pull 9912, but the |
Beta Was this translation helpful? Give feedback.
-
|
@MarcA711 Thanks, I'll stay tuned |
Beta Was this translation helpful? Give feedback.
-
|
Hey @XMaarten, Yes, see my comment here #9912 (comment). I have to either disable afbc completely or find a way to enable it optionally. |
Beta Was this translation helpful? Give feedback.
-
|
I updated the presets now in my forked repo: https://github.com/MarcA711/frigate-rockchip Please help me to test the changes and give feedback if everything works and if your cpu load increases or decreases. If everything works, I will merge the new presets. You can either build from source using |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Works perfect for me, CPU decreased and is around 3 % per camera. |
Beta Was this translation helpful? Give feedback.
-
|
I made changes in docker by this commit, both cameras work fine, both cameras are h264 streams |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I found a new problem, if I scale the resolution to the original resolution 16:9, the debug screen works fine, but if I don't scale the original ratio such as 4:3 or whatever, the screen will appear blurred config 
|
Beta Was this translation helpful? Give feedback.
9 replies
-
|
@nyanmisaka I encountered a similar problem when testing jellyfin, transcoding 4K VP9 into 1080P H264 when the screen is blurred, I don't know if this is the same problem, the test video docker-compose 
|
Beta Was this translation helpful? Give feedback.
-
|
Can you share a ffmpeg trancoding log so I know what's going on? |
Beta Was this translation helpful? Give feedback.
-
|
@nyanmisaka Thanks, it's too late today, if there is anything I need to reply in time tomorrow morning |
Beta Was this translation helpful? Give feedback.
-
|
RK356x supports 1080p encoding, and the VP8 decoder supports up to 1080p decoding. It's clear that rockchip doesn't support 4k VP8 decoding. Therefore this resolution (4000x2250) is out of specs. In the future we may impose some restrictions to prevent it from using hardware codecs. |
Beta Was this translation helpful? Give feedback.
-
Thx I see |
Beta Was this translation helpful? Give feedback.
-
|
Frigate still says "Hardware acceleration has not been setup" in the GUI, but |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@MarcA711 Please add this commit to fix the issue of scaling different proportions of the green screen |
Beta Was this translation helpful? Give feedback.
0 replies
-
You need to use this docker image and according to the configuration I said at the beginning, download the model and put it in the corresponding directory, and you also need to enter the system to see if the npu and vpu are correctly enabled 
|
Beta Was this translation helpful? Give feedback.
3 replies
-
|
@liwei19920307 you don't need to download a model and put it in some folder. You can just set the model in your frigate config like this and frigate will download the model automatically: |
Beta Was this translation helpful? Give feedback.
-
|
Okay, because there is GFW, many times it is more convenient to download manually. Unless you use a proxy service, but it's also cumbersome to give docker a proxy There is also a question, if you need to download automatically, I would like to know if this model is same to the configuration you recommend Does the RK3566 need to be configured like this? Or is it okay to just write it like this? |
Beta Was this translation helpful? Give feedback.
-
|
I see, if I press your configuration, the frigate in docker will automatically download the corresponding model to /config/model_cache/rknn, and it doesn't seem to need a proxy anymore config model in docker root@ha-green:/config/model_cache/rknn# ls
yolov8s-320x320-rk3566.rknn |
Beta Was this translation helpful? Give feedback.
-
|
I merged a new version that allows changing the aspect ratio during decoding. Also pushed a new docker image.
@XMaarten I never realized that message, but I also got it. When I have more time, I will try to fix this, so it correctly displays that rockchip hw aceeleration is used. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I tested |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I test My detectors config: If I set |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
hello everyone, do this or these solutions work with a Green home assistant please? and if so what is the procedure to follow? |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
This only applies to the system of the vendor kernel, unfortunately the official OS of HA is based on the mainline kernel, if you are familiar with linux, you can try the armbian I compiled, and use it after booting it through the SD card. Need to use docker to run frigate. For more tutorials, check out my wiki, I'm from China and may need your own translation |
Beta Was this translation helpful? Give feedback.
-
|
thank you very much for your response, I will look at your wiki in detail! |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
nyanmisaka/ffmpeg-rockchip#26 (comment)