[Detector Support]: rknn is slow to respond.

[n3o4po11o]

Describe the problem you are having

I'm using rk3566 with rknn using default yolov8n model
Inference Speed: 61.83 ms

I'm also have Intel N100 with openvino using ssdlite_mobilenet_v2.xml
Inference Speed: 13.5 ms

It takes about 2 or 3 second for rknn start to output detect results. So If I leave detect zone within 2 second. The event will be empty. At the same time openvino detect every time.

@MarcA711

Version

frigate-rockchip-0.13.2-788D6B17

Frigate config file

mqtt:
  enabled: True
  # Required: host name
  host: 192.168.60.5
  # Optional: port (default: shown below)
  port: 1883
  # Optional: topic prefix (default: shown below)
  # NOTE: must be unique if you are running multiple instances
  topic_prefix: frigate
  # Optional: client id (default: shown below)
  # NOTE: must be unique if you are running multiple instances
  client_id: frigate
  # Optional: user
  # NOTE: MQTT user can be specified with an environment variables that must begin with 'FRIGATE_'.
  #       e.g. user: '{FRIGATE_MQTT_USER}'
  user: fallen
  # Optional: password
  # NOTE: MQTT password can be specified with an environment variables that must begin with 'FRIGATE_'.
  #       e.g. password: '{FRIGATE_MQTT_PASSWORD}'
  password: x
go2rtc:
  streams:
    balcony:
      rtsp://admin:[email protected]:554/Streaming/Channels/101
  webrtc:
    candidates:
      - 192.168.60.13:8555
      - stun:8555

cameras: 
  frontdoor: 
    record: 
       enabled: true
       retain:
         days: 9999
         mode: all
       events:
         retain:
           default: 9999
           mode: active_objects
         required_zones:
           - frontdoor_corridor
           - frontdoor_entire
    snapshots:
      required_zones:
        - frontdoor_corridor
        - frontdoor_entire
    ffmpeg: 
      output_args: 
        record: preset-record-generic-audio-aac
      inputs: 
        - path: rtsp://rtsp:[email protected]:554/Streaming/Channels/102
          input_args: preset-rtsp-generic
          hwaccel_args: preset-rk-h264 
          roles: 
            - detect #检测，用来分析的流，不要选择h265，而且一般摄像头那边设置5fps也够了，720p最高，可以调低
        - path: rtsp://rtsp:[email protected]:554/Streaming/Channels/101
          input_args: preset-rtsp-generic
          hwaccel_args: preset-rk-h265 #此时摄像头可以选择主流h265编码，最好是可变码率，这样节约空间
          roles: #选择角色
            - record #用于保存视频
    zones:
      frontdoor_corridor:
        coordinates: 567,367,132,0,0,0,0,346,201,480,500,607
      frontdoor_entire:
        coordinates: 205,0,1280,0,1280,525,1280,720,731,720,0,720,0,432,589,675,662,376
    live:
       stream_name: frontdoor_cam_sub
detect:
  width: 640
  height: 360
  fps: 6
  enabled: True
  # max_disappeared: 25
  # stationary:
  #   interval: 1
  #   threshold: 50
  #   max_frames:
  #     default: 3000
  #     objects:
  #       person: 1000
objects:
  track:
    - person
  filters:
    person:
      min_score: 0.2

detectors:
  rknn:
    type: rknn

motion:
    mask:
      - 1145,0,1148,31,760,36,754,0
model:
  #path: /config/model_cache/rknn/yolov8n-320x320-rk3588-custom.rknn
  path: default-yolov8x
  input_pixel_format: bgr
  width: 320
  height: 320
  input_tensor: nhwc

logger:
  # Optional: default log level (default: shown below)
  default: debug
  # Optional: module by module log level configuration
  logs:
    # frigate.mqtt: debug
    ffmpeg.frontdoor.detect: debug
    detect: debug
    # level: debug  # default level
    # api: debug
    # exec: debug
    # ngrok: debug
    # rtsp: debug
    # streams: debug
    # webrtc: debug

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    stop_grace_period: 1s
    image: frigate-rockchip:0.13.2-788D6B17
    shm_size: "256mb" # update for your cameras based on calculation above
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/frigate/config/:/config/
      - /opt/frigate/dbs/:/frigate/dbs/
      - /opt/frigate_media/:/media/frigate #将frigate持久化数据文件夹挂载进去
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password"

Relevant log output

None

Operating system

Debian

Install method

Docker Compose

Coral version

Other

Any other information that may be helpful

No response

[MarcA711]

Hey,
your detect resolution is 640x360. Could you try to increase it to 1280x720? I remember that I had some similar problems when I tested a lower resolution in my setup. If the low resolution causes this weird behavior, I can try to find the issue.

[MarcA711]

Can somebody reproduce this issue?

I plan to upgrade to a new version of the rknn toolkit soon, maybe this will fix your issues?

[n3o4po11o]

Open Debug page on this camera.

Before start, ensure this feed don't trigger any motion event, the NPU usage would be 0%

Entering zone, see how long it takes to show "Person %" in debug page

[n3o4po11o]

I would check after you upgrade to 1.6.0

[n3o4po11o]

I check the 1.6.0 from MarcA711@69d9a26

The same result

[MarcA711]

After further investigation, I can reproduce the behavior that you describe. However, I have no clue what causes this issue. I will update, if I find out more details. Thank you really much for reporting!