[HW Accel Support]: Intel iGPU + ARC GPU

[magma1447]

Describe the problem you are having

I am running Frigate with docker compose. My system has an iGPU and also an ARC GPU.

If I start Frigate with the iGPU only everything works fine. I prevent the ARC GPU from being used by echo "0000:03:00.0" | sudo tee /sys/bus/pci/drivers/i915/unbind. As soon as I bind that device again Frigate fails to start (crashes). See Relevant Frigate log output below.

The host system is Debian stable. With a kernel (6.11.10) from backports since the ARC A310 wasn't supported by Debian Stable's kernel (6.1, 6.3 was recommended, if I recall correctly).

If I instead disable the iGPU and enable the ARC a similar crash of Frigate happens.

If I run clinfo inside the container one platform is shown if having the iGPU only. ARC only or both ends with zero platforms returned.

My end goal is to provide two GPUs that can be used for detectors. Anyone having more clue than I have? Or should I just assume that it's an issue with the Intel drivers?

Version

0.14.1-f4f3cfa

Frigate config file

detectors:
  ov:
    type: openvino
    device: GPU

model:
  model_type: yolonas
  width: 320
  height: 320
  input_tensor: nchw
  input_pixel_format: bgr
  path: /config/models/yolo_nas_m.onnx
  labelmap_path: /labelmap/coco-80.txt

docker-compose file or Docker CLI command

services:
  frigate:
    container_name: frigate
    network_mode: bridge
    privileged: true
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128 # passes GPU
      - /dev/dri/renderD129:/dev/dri/renderD129 # passes GPU
    
    shm_size: "384MB"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./volumes/config:/config
      - media_nfs01:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: 1073741824
    ports:
      - "5000:5000"
      - "9554:8554" #RTSP feeds
      - "9555:8555/tcp" # WebRTC over tcp
      - "9555:8555/udp" # WebRTC over udp
    environment:
      - TZ=Europe/Stockholm
      - FRIGATE_RTSP_PASSWORD="REDACTED"
volumes:
  media_nfs01:
    driver: local
    driver_opts:
      type: nfs
      o: "addr=REDACTED,nfsvers=4,soft,noexec,nosuid,tcp,rw"
      device: ":/srv/frigate_media"

Relevant Frigate log output

frigate  | 2024-12-30 18:15:33.540557222  Process detector:ov:
frigate  | 2024-12-30 18:15:33.541431447  Traceback (most recent call last):
frigate  | 2024-12-30 18:15:33.541434649    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
frigate  | 2024-12-30 18:15:33.541436112      self.run()
frigate  | 2024-12-30 18:15:33.541437585    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
frigate  | 2024-12-30 18:15:33.541441610      self._target(*self._args, **self._kwargs)
frigate  | 2024-12-30 18:15:33.541443256    File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
frigate  | 2024-12-30 18:15:33.541460205      object_detector = LocalObjectDetector(detector_config=detector_config)
frigate  | 2024-12-30 18:15:33.541462324    File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
frigate  | 2024-12-30 18:15:33.541463555      self.detect_api = create_detector(detector_config)
frigate  | 2024-12-30 18:15:33.541464709    File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
frigate  | 2024-12-30 18:15:33.541482579      return api(detector_config)
frigate  | 2024-12-30 18:15:33.541484162    File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 43, in __init__
frigate  | 2024-12-30 18:15:33.541485447      self.interpreter = self.ov_core.compile_model(
frigate  | 2024-12-30 18:15:33.541486662    File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 521, in compile_model
frigate  | 2024-12-30 18:15:33.541505010      super().compile_model(model, device_name, {} if config is None else config),
frigate  | 2024-12-30 18:15:33.541506277  RuntimeError: Exception from src/inference/src/cpp/core.cpp:123:
frigate  | 2024-12-30 18:15:33.541507451  Exception from src/inference/src/dev/plugin.cpp:59:
frigate  | 2024-12-30 18:15:33.541508687  Check 'contexts.count(device_id)' failed at src/plugins/intel_gpu/src/plugin/plugin.cpp:204:
frigate  | 2024-12-30 18:15:33.541510079  [GPU] Context was not initialized for 0 device

Relevant go2rtc log output

N/A

FFprobe output from your camera

N/A

Install method

Docker Compose

Object Detector

OpenVino

Network connection

Wired

Camera make and model

Several Hikvisions

Screenshots of the Frigate UI's System metrics pages

N/A since Frigate doesn't start.

Any other information that may be helpful

No response

[NickM-27]

So does it work if you limit to one or the other via the compose config?

[magma1447]

That's the weird part, at least to me. They both show up in the container even though I remove them both.

services:
  frigate:
    container_name: frigate
    network_mode: bridge
    privileged: true
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    #build:
    #  context: image
    #devices:
      #- /dev/bus/usb:/dev/bus/usb  # Passes the USB Coral
      #- /dev/dri/renderD128:/dev/dri/renderD128 # passes GPU
      #- /dev/dri/renderD129:/dev/dri/renderD129 # passes GPU

docker compose down
vim docker-compose.yaml
docker compose up -d
docker exec -it frigate bash
root@e07748ae567a:/opt/frigate# ls -l /dev/dri
total 0
crw-rw---- 1 root video 226,   0 Dec 31 09:33 card0
crw-rw---- 1 root video 226,   1 Dec 31 09:33 card1
crw-rw---- 1 root   105 226, 128 Dec 31 09:33 renderD128
crw-rw---- 1 root video 226, 129 Dec 31 09:33 renderD129

That's why I disabled the ARC like this:
echo "0000:03:00.0" | sudo tee /sys/bus/pci/drivers/i915/unbind

Also, even if none of them are included in docker-compose.yaml, and the ARC is unbind the iGPU actually works. sudo intel_gpu_top -d drm:/dev/dri/card0 shows that it's getting used. Just as shown above the dri device exists in the container even though I don't pass it.

I am using Docker's offical repositories.
deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian bookworm stable

The operating system is a new fresh install. I have moved my Frigate installation (docker-compose.yaml) from an old (temporary) NUC to new hardware.

[magma1447]

I just realized that the dri devices gets passed because I have privileged: true. I don't recall why I have that though. Just removed it.
EDIT: I assume I had it because the installation instructions says to use it while settings things up and testing. Then I probably forgot about it.

• If only passing render128 things works.
• If passing both, same error as I originally posted.
• If only passing render129 as render129, same error as originally posted.
• If only passing render129 as render128, same error as originally posted.

[magma1447]

I notice that https://github.com/blakeblackshear/frigate/blob/dev/docker/main/install_deps.sh has been updated regarding Intel. Maybe it would be worth for me to try 0.15.0 Beta 4 to see if that makes a difference.

Frigate's stable release is using intel-opencl-icd version 22.43.24595.41-1. Seems to be 24.35.30872.31-996~22.04 in 0.15. I will test this later today.

[magma1447]

Okay. This is silly (by me). I upgraded to Frigate v0.15-beta4. Now it works. I have spent ~20 hours trying to test different things with the Frigate container. Building new drivers, building new containers and whatnot. And all that was required was a 10 byte change. :)

Thanks for taking the time to answer me @NickM-27 . This will be interesting, to see if the A310 is enough. Due to the choice of hardware I was a bit limited to which GPU I could choose.

[ProsjektX]

What kind of inference speeds are you getting with the A310 with openvino/yolonas?

[magma1447]

What kind of inference speeds are you getting with the A310 with openvino/yolonas?

These are the numbers I wrote down from when there isn't any real action.

• 20-25 ms with yolo-nas M (320x320).
• 30-35 ms with yolo-nas L (320x320).

I (obviously) just got the A310 running, so I don't have much details or experience with it yet. What's weird to me is that intel-gpu-top shows the engine using the A310 as [unknown] and it also peaks at 100% quite often. While Detector CPU Usage in Frigate's web-ui stays around 3x3.5% (I have three detectors configured).

I haven't been able to run it with 640x640. I thought it would work if I reduced to a single detector, but it doesn't. I assume it's due to the low amount of memory in the A310.

2024-12-31 13:22:43.509488158  Process camera_processor:cam-13:
2024-12-31 13:22:43.509490070  Traceback (most recent call last):
2024-12-31 13:22:43.509490944    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-12-31 13:22:43.509491732      self.run()
2024-12-31 13:22:43.509492510    File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
2024-12-31 13:22:43.509493212      return run(*args, **kwargs)
2024-12-31 13:22:43.509493989    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-12-31 13:22:43.509494721      self._target(*self._args, **self._kwargs)
2024-12-31 13:22:43.509495431    File "/opt/frigate/frigate/video.py", line 440, in track_camera
2024-12-31 13:22:43.509496058      object_detector = RemoteObjectDetector(
2024-12-31 13:22:43.509496821    File "/opt/frigate/frigate/object_detection.py", line 216, in init
2024-12-31 13:22:43.509497516      self.np_shm = np.ndarray(
2024-12-31 13:22:43.509498191  TypeError: buffer is too small for requested array
2024-12-31 13:22:43.511599791  [2024-12-31 13:22:43] frigate.app                    INFO    : Capture process started for cam-12-ptz: 600
2024-12-31 13:22:43.513529307  Process camera_processor:cam-14:
2024-12-31 13:22:43.513532871  Traceback (most recent call last):
2024-12-31 13:22:43.513534414    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-12-31 13:22:43.513535535      self.run()
2024-12-31 13:22:43.513536746    File "/opt/frigate/frigate/util/process.py", line 41, in run_wrapper
2024-12-31 13:22:43.513537726      return run(*args, **kwargs)
2024-12-31 13:22:43.513538880    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-12-31 13:22:43.513539912      self._target(*self._args, **self._kwargs)
2024-12-31 13:22:43.513541110    File "/opt/frigate/frigate/video.py", line 440, in track_camera
2024-12-31 13:22:43.513542081      object_detector = RemoteObjectDetector(
2024-12-31 13:22:43.513543185    File "/opt/frigate/frigate/object_detection.py", line 216, in init
2024-12-31 13:22:43.513544036      self.np_shm = np.ndarray(
2024-12-31 13:22:43.513544971  TypeError: buffer is too small for requested array

I would have bought a B580 if there was room enough for it. But there wasn't. I needed single unit of height and a bit shorter card.

EDIT: It's 4k Hikvision cameras with 16 Mbps VBR. Detection stream is 720p/6 fps.