[Detector Support]: High Detector CPU load with Hailo-8L

[chrasim1]

Describe the problem you are having

I recently changed from a Google Coral on a J4125 to a Hailo-8L on an N150, because the Hailo was easier available and I supposed that a bit more power is better.

However, now I see that with the Hailo the "Detector CPU Load" is much bigger. I have both servers running in parallel with the same cameras and streams, detector cpu load on the Coral is currently 11,1% while on the hailo it's 34,1%. When it's dark outside, the hailo goes up to 80% and starts missing frames.

I have 7 cameras, detection is running on substreams with 1280x720. I'm using the standard detector models. Why is the CPU load so much highter with the Hailo? With OpenVino on the GPU I see CPU loads similar to using the Hailo.

Version

0.16.1-e664cb2

Frigate config file

mqtt:
  host: 192.168.12.33
  user: mqtt
  password: crMqtt1!

motion:
  improve_contrast: true

#detectors:
#  coral:
#    type: edgetpu
#    device: pci
#detectors:
# ov_0:
#    type: openvino
#    device: GPU
# ov_1:
#    type: openvino
#    device: GPU
#
#model:
#  width: 300
#  height: 300
#  input_tensor: nhwc
#  input_pixel_format: bgr
#  path: /openvino-model/ssdlite_mobilenet_v2.xml
#  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

detectors:
  hailo:
    type: hailo8l
    device: PCIe

model:
  width: 320
  height: 320
  input_tensor: nhwc
  input_pixel_format: rgb
  input_dtype: int
  model_type: yolo-generic
  labelmap_path: /labelmap/coco-80.txt
  
birdseye:
  restream: true
  enabled: true
  width: 1280
  height: 720
  quality: 8

ffmpeg:
  #hwaccel_args: preset-vaapi
  hwaccel_args: preset-intel-qsv-h264

go2rtc:
  streams:
    garagentor:
      - ffmpeg:rtsp://admin:[email protected]/cam/realmonitor?channel=1&subtype=0#backchannel=0
    garagentor_hd:
      - ffmpeg:rtsp://admin:[email protected]/cam/realmonitor?channel=1&subtype=1#backchannel=0
    garten_sub:
      - rtsp://dahua:[email protected]:554/cam/realmonitor?channel=1&subtype=2&unicast=true&proto=Onvif # <- stream which supports video & aac audio. This is only supported for rtsp streams, http must use ffmpeg
      # - "ffmpeg:rtsp_cam#audio=opus" # <- copy of the stream which transcodes audio to opus
    garten:
      - rtsp://dahua:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif # <- stream which supports video & aac audio. This is only supported for rtsp streams, http must use ffmpeg     
    hinten_sub:
      - rtsp://admin:[email protected]:554/H264/ch1/sub/av_stream
    hinten:
      - rtsp://admin:[email protected]:554/H264/ch1/main/av_stream
    kinderzimmer_sub:
      - rtsp://admin:[email protected]:554/H264/ch1/sub/av_stream
    kinderzimmer:
      - rtsp://admin:[email protected]:554/H264/ch1/sub/av_stream
    vorne_sub:
      - rtsp://admin:[email protected]:554/H264/ch1/sub/av_stream
    vorne:
      - rtsp://admin:[email protected]:554/H264/ch1/main/av_stream
    haustuer_sub:
      - rtsp://admin:[email protected]:554/H264/ch1/sub/av_stream
    haustuer:
      - rtsp://admin:[email protected]:554/H264/ch1/main/av_stream
    test_sub:
      - rtsp://admin:[email protected]:554/H264/ch1/sub/av_stream
    test:
      - rtsp://admin:[email protected]:554/H264/ch1/main/av_stream

cameras:
  Garten:  # <------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garten_sub # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/garten
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
    live:
      streams:
        garten: garten
  Garagentor:  # <------ Name the camera
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/garagentor
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/garagentor_hd
          input_args: preset-rtsp-restream
          roles:
            - record
      #output_args:
      #  record: preset-record-generic-audio-copy
        #- path: rtsp://127.0.0.1:8554/garagentor # <--- the name here must match the name of the camera in restream
        #  input_args: preset-rtsp-restream
        #- path: rtsp://dahua:[email protected]:554/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
        #  roles:
        #    - detect
        #    - record
    detect:
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
    live:
      streams:
        garagentor: garagentor
  Hinten:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/hinten_sub # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/hinten # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
    live:
      streams:
        hinten: hinten
  Kinderzimmer:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/kinderzimmer_sub # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/kinderzimmer # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
    live:
      streams:
        kinderzimmer: kinderzimmer
  Vorne:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/vorne_sub # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/vorne # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record
     # output_args:
      #  detect: '-vf scale=854:480 -f rawvideo -pix_fmt yuv420p'
    detect:
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
    live:
      streams:
        vorne: vorne
    objects:
      filters:
        person:
          mask: 0.444,0.04,0.442,0.132,0.511,0.152,0.501,0.038
  Haustuer:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/haustuer_sub # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/haustuer # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record
     # output_args:
      #  detect: '-vf scale=854:480 -f rawvideo -pix_fmt yuv420p'
    detect:
      width: 1280 # <---- update for your camera's resolution
      height: 720 # <---- update for your camera's resolution
    live:
      streams:
        haustuer: haustuer
record:
  enabled: true
  retain:
    days: 0
  alerts:
    retain:
      days: 5
  detections:
    retain:
      days: 5
snapshots:
  # Optional: Camera override for retention settings (default: global values)
  retain:
    # Required: Default retention days (default: shown below)
    default: 5
    # Optional: Per object retention days
    objects:
      person: 5

version: 0.16-0
detect:
  enabled: true
  fps: 5
objects:
  track:
    - person
semantic_search:
  enabled: false
  model_size: small
face_recognition:
  enabled: true
  model_size: small
  # min_area: 300
lpr:
  enabled: false
classification:
  bird:
    enabled: false
ui:
  time_format: 24hour
  date_style: short

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    stop_grace_period: 30s # allow enough time to shut down the various services
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "512mb" # update for your cameras based on calculation above
    devices:
      - /dev/hailo0
      - /dev/apex_0:/dev/apex_0 # Passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/dri/renderD128:/dev/dri/renderD128 # For intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - CHANGE_TO_COMPOSE_DATA_PATH/frigate/config:/config
      - CHANGE_TO_COMPOSE_DATA_PATH/frigate/media:/media
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "8971:8971"
      - "5000:5000" # Internal unauthenticated access. Expose carefully.
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "xxx"

Relevant Frigate log output

No error messages in the log.

Install method

Docker Compose

Object Detector

Other

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[inkeep-ai-assistant[bot]]

The higher CPU load with your Hailo-8L compared to the Google Coral is a known behavior difference between these detector types. Here's what's happening and some potential solutions:

Why Hailo Shows Higher CPU Load

The Hailo-8 detector requires more CPU preprocessing compared to the Coral EdgeTPU (1). The Coral EdgeTPU is specifically designed for TensorFlow Lite models and has optimized preprocessing pipelines, while the Hailo detector uses a different architecture that can result in higher CPU overhead for frame preparation and post-processing (1).

Configuration Optimizations

Your current Hailo configuration is using a 320x320 model, but you might benefit from adjusting the model parameters. The Hailo detector supports both YOLO and SSD model types (1). For your setup with 7 cameras at 1280x720 detection resolution, consider these optimizations:

1. Try the SSD model instead of YOLO-generic:

detectors:
  hailo:
    type: hailo8l
    device: PCIe

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: rgb
  model_type: ssd

(1)

2. Consider using multiple detectors if your hardware can support it:

detectors:
  hailo_0:
    type: hailo8l
    device: PCIe
  hailo_1:
    type: hailo8l
    device: PCIe

(1)

Hardware Considerations

The Intel N150 processor you're using has limited processing power compared to more modern Intel platforms (2). The documentation shows that Intel N150 has inference times around 15-16ms for various models, which may contribute to the higher CPU load when handling 7 cameras simultaneously (2).

Alternative Solutions

Given your high CPU usage, you might want to consider:

1. Reduce detection resolution - Instead of 1280x720, try 854x480 or 640x360 for the detection stream
2. Lower detection FPS - Your current 5 FPS could be reduced to 3-4 FPS during high-load periods
3. Use zones and masks more aggressively to reduce the detection area

The behavior you're experiencing aligns with community reports of Hailo detectors requiring more CPU resources compared to Coral devices, particularly on lower-powered Intel platforms (3).

📚 Sources:

1. Supported Hardware
2. Recommended hardware
3. [Detector Support]: Slow Inference & High CPU Usage on RK3568 with Hailo-8 Accelerator

---

Have another question? Just tag @inkeep.

[hawkeye217]

The AI is correct here. The Detector CPU Usage metric is measuring the preprocessing steps to prepare the model inputs to run inference on your detector. With more complex models that the Hailo runs compared to Coral, this is expected.

I would try tuning your motion detection for all your cameras to try to reduce the number of frames sent off to the object detector. See the docs:

https://docs.frigate.video/configuration/motion_detection

[chrasim1]

Thanks, but since I see similar Interference, CPU load and system power consumption values for Hailo-8L and OpenVino GPU and from what I understand both run similar models, it seems that the Hailo does not make sense for this kind of system at all and I should rather optimize OpenVINO or get a coral, correct?

[hawkeye217]

With your N150, OpenVINO can be used for both object detection and enrichments (face recognition, LPR, Semantic Search) so it's an all-around better choice than a device that doesn't support OpenVINO (enrichments can't run on a Hailo or Coral).

But since you're not using any enrichments outside of Semantic Search (with the small model, which doesn't run on a GPU anyways), it doesn't seem like the Hailo is giving you any benefit over OpenVINO.

I'd definitely try tuning your motion detection in any case to see if you can reduce the Detector CPU Usage further.

[NickM-27]

To be clear they do not run similar models. The hailo runs YOLOv6n by default which is a larger and more advanced model than the ssdlite that the coral runs.

[hawkeye217]

Right, I think the user was noting the similarity of models between Hailo and OpenVINO, not Hailo and Coral.