[Detector Support]: Openvino and Intel integrated NPU support #13248
-
Describe the problem you are havingHello, I have been using Frigate 14 with YoloNAS-M (256x256) with great satisfaction. For my use case, I have no FP and an inference speed below 10ms with an Intel Core Ultra 7 155H with the iArc GPU. I have been exploring the possibility to utilize the NPU, to offload the GPU from this task 1. I have sucessfully downloaded the NPU drivers from : 2. I have an initialized VPU with : 3. I also have an accel0 device passthrough working: 4. When I try to set the detector device to NPU in the Frigate config, I get the following message (see log below): 5. However, when looking inside the container, the plugin seems present: I observe the same behavior when using : model: Has anyone tried the integrated Intel NPU with Openvino before ? Many thanks, Philippe Version0.14.0-da913d8 Frigate config filemqtt:
host: 192.168.5.252
port: 1883
user: xxxx
password: xxxx
topic_prefix: frigate
cameras:
Cam_Garage: # <------ Name the camera
ui:
order: 3
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path:
rtsp://xxxx:[email protected]:554/axis-media/media.amp?videocodec=h264&resolution=800x450&fps=5
roles:
- detect
- path:
rtsp://xxxx:[email protected]:554/axis-media/media.amp?videocodec=h264&resolution=1920x1080&fps=5
roles:
- record
zones:
entire_garage:
inertia: 3
coordinates: 0,450,800,450,800,187,528,50,284,21,0,55
detect:
enabled: true
width: 800
height: 450
fps: 5
motion:
mask: 0.356,0.043,0.663,0.109,1,0.403,1,0,0,0,0,0.117
improve_contrast: true
objects:
track:
- person
- cat
- dog
filters:
person:
threshold: 0.65 # 0.70 for Yolov8s_srt_100eV3. 0.6 for Yolov8s
min_score: 0.55
min_ratio: 0.20
max_ratio: 0.8
snapshots:
required_zones:
- entire_garage
enabled: true
bounding_box: true
retain:
default: 90
record:
enabled: true
retain:
days: 0
# mode: active_objects
events:
retain:
default: 30
mode: active_objects
review:
alerts:
required_zones:
- entire_garage
Cam_Garden: # <------ Name the camera
ui:
order: 1
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path:
rtsp://xxxx:[email protected]:554/axis-media/media.amp?videocodec=h265&resolution=800x600&fps=5
roles:
- detect
- path:
rtsp://xxxx:[email protected]:554/axis-media/media.amp?videocodec=h265&resolution=1600x1200&fps=5
roles:
- record
zones:
entire_garden:
inertia: 3
coordinates: 0,1,0.71,1,0.835,0.8,0.963,0.474,0.57,0.267,0.287,0.15,0,0.307
detect:
enabled: true
width: 800
height: 600
fps: 5
motion:
mask: 0.29,0.141,0.58,0.264,0.695,0.322,1,0.485,1,0,0.278,0,0,0,0,0.296,0.162,0.212
improve_contrast: true
threshold: 30
contour_area: 10
objects:
track:
- person
- cat
- dog
filters:
person:
threshold: 0.65
min_score: 0.55
min_area: 250
max_area: 10000
snapshots:
required_zones:
- entire_garden
enabled: true
bounding_box: true
retain:
default: 90
record:
enabled: true
retain:
days: 0
# mode: active_objects
events:
retain:
default: 30
mode: active_objects
review:
alerts:
required_zones: entire_garden
Cam_Cave: # <------ Name the camera
ui:
order: 2
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path:
rtsp://xxxx:[email protected]:554/axis-media/media.amp?videocodec=h265&resolution=800x600&fps=5
roles:
- detect
- path:
rtsp://xxxx:[email protected]:554/axis-media/media.amp?videocodec=h265&resolution=1600x1200&fps=5
roles:
- record
zones:
entire_cave:
inertia: 3
coordinates: 246,600,800,600,756,600,720,349,516,208,223,100,102,108,139,300,105,315,138,405
detect:
enabled: true
width: 800
height: 600
fps: 5
motion:
mask:
- 156,281,135,163,279,162,462,211,800,365,800,0,0,0,0,331
improve_contrast: true
threshold: 20
objects:
track:
- person
- cat
- dog
filters:
person:
threshold: 0.65
min_score: 0.55
min_area: 200
max_area: 14000
mask:
0.17,0.272,0.369,0.278,0.372,0.339,0.384,0.339,0.387,0.289,0.573,0.355,1,0.606,1,0,0,0.002,0,0.552,0.196,0.466
snapshots:
required_zones:
- entire_cave
enabled: true
bounding_box: true
retain:
default: 90
record:
enabled: true
retain:
days: 0
# mode: active_objects
events:
retain:
default: 30
mode: active_objects
review:
alerts:
required_zones:
- entire_cave
Cam_Entree: # <------ Name the camera
ui:
order: 4
ffmpeg:
hwaccel_args: preset-vaapi
inputs:
- path:
rtsp://xxxx:[email protected]:554/axis-media/media.amp?videocodec=h264&resolution=640x480&fps=5
roles:
- detect
- path:
rtsp://xxxx:[email protected]:554/axis-media/media.amp?videocodec=h264&resolution=640x480&fps=5
roles:
- record
detect:
enabled: true
width: 640
height: 480
fps: 5
objects:
track:
- person
- cat
- dog
filters:
person:
threshold: 0.65
min_score: 0.55
mask: 1,0.357,1,0,0,0,0,0.321,0.593,0.325,0.601,0.346
snapshots:
enabled: true
bounding_box: true
retain:
default: 90
record:
enabled: true
retain:
days: 0
# mode: active_objects
events:
retain:
default: 30
mode: active_objects
motion:
mask: 0.001,0.002,0.999,0.001,0.999,0.355,0.602,0.346,0.592,0.322,0,0.32
threshold: 40
contour_area: 10
improve_contrast: true
detectors:
ov:
type: openvino
device: NPU
model:
path: /models/yolo_nas_m_256.onnx
model:
model_type: yolonas
width: 256 # <--- should match whatever was set in notebook
height: 256 # <--- should match whatever was set in notebook
input_tensor: nchw
input_pixel_format: bgr
labelmap_path: /models/coco_80cl.txt
version: 0.14
camera_groups: {}docker-compose file or Docker CLI commandversion: "3.9"
services:
frigate:
container_name: frigate
privileged: true # this may not be necessary for all setups
restart: unless-stopped
image: ghcr.io/blakeblackshear/frigate:0.14.0
shm_size: "256mb" # update for your cameras based on calculation above
devices:
- /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
- /dev/dri
- /dev/accel
volumes:
- /etc/localtime:/etc/localtime:ro
- /root/frigate/config/:/config/
- /root/frigate:/media/frigate
- /root/frigate/models:/models
- type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
target: /tmp/cache
tmpfs:
size: 1000000000
ports:
- "15000:5000"
- "19350:1935" # RTMP feeds
environment:
PLUS_API_KEY: xxxx
NEOReadDebugKeys: 1
OverrideGpuAddressSpace: 48Relevant log output2024-08-21 09:28:04.483711629 Process detector:ov:
2024-08-21 09:28:04.484450278 Traceback (most recent call last):
2024-08-21 09:28:04.484477079 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
2024-08-21 09:28:04.484479530 self.run()
2024-08-21 09:28:04.484480738 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
2024-08-21 09:28:04.484485230 self._target(*self._args, **self._kwargs)
2024-08-21 09:28:04.484487045 File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
2024-08-21 09:28:04.484500125 object_detector = LocalObjectDetector(detector_config=detector_config)
2024-08-21 09:28:04.484501349 File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
2024-08-21 09:28:04.484502754 self.detect_api = create_detector(detector_config)
2024-08-21 09:28:04.484503632 File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
2024-08-21 09:28:04.484504447 return api(detector_config)
2024-08-21 09:28:04.484505400 File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 43, in __init__
2024-08-21 09:28:04.484519659 self.interpreter = self.ov_core.compile_model(
2024-08-21 09:28:04.484520992 File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 521, in compile_model
2024-08-21 09:28:04.484531392 super().compile_model(model, device_name, {} if config is None else config),
2024-08-21 09:28:04.484540486 RuntimeError: Exception from src/inference/src/cpp/core.cpp:123:
2024-08-21 09:28:04.484542622 Cannot load library '/usr/local/lib/python3.9/dist-packages/openvino/libs/libopenvino_intel_npu_plugin.so': libze_loader.so.1: cannot open shared object file: No such file or directoryOperating systemProxmox Install methodDocker Compose Object DetectorOpenVino Any other information that may be helpfulAsus NUC Intel Core Ultra 7 155H with iArc GPU. |
Beta Was this translation helpful? Give feedback.
Replies: 9 comments 43 replies
-
|
Bumping up this one.... |
Beta Was this translation helpful? Give feedback.
-
|
Hi gyus. I have solved this issue installing the NPU drivers and from intel and updated OpenVino to the ltest version using pip and even once i got NPU working. Unfortunately I didn't get any screenshots to show how it works. CPU is Ultra 5 125H I've installed all the drivers inside the docker container but didn't figure out how to commit changes and lauch it as the HA addon. So after container restart i just lost all progres, but for my opinion this would be great to add NPU support. It is really easy, working out of the box. And NPU can be really useful for some specific cases. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks cheese202 for your comments. A least you got it to work. So my issue must be linked to my setup. I will go back to it next month and see what I can do. |
Beta Was this translation helpful? Give feedback.
-
|
I'm having the same issue. ASUS nuc14 pro with Intel Core Ultra 7 155H processor. frigate log
frigate config
docker compose yaml
I seem to have no problem accessing libopenvino_intel_npu_plugin.so from inside the container (docker exec)
Is anyone else having the same issue and has found a solution? |
Beta Was this translation helpful? Give feedback.
-
|
openvino has been updated for 0.15 so may work better there |
Beta Was this translation helpful? Give feedback.
-
|
So i can confirm that intel NPU is works fine for me. There are the steps how i got it works.
So i'm not a pro in github usage but i'll try to create the pull request to add intel NPU drivers in dev container because for this you just need to install one package from debial bookwork and 4 packages from intel. The main 2 problems which i faced with are:
also it seems that detector CPU load shows incorrect data because htop shows host load no more than 18% So it would be awesome to add NPU support because it can be very potential for some cases and also feature mentioned somewhere there to add the possibility to use separate detector for specific camera |
Beta Was this translation helpful? Give feedback.
-
It's showing correct data, it's calculated inside the container as the % of a single core |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @cheese202, I managed to get the NPU work using your last post. Like you I can run the integrated ssdlite model, but not the yolonas models (frigate+ trained or COCO trained). |
Beta Was this translation helpful? Give feedback.
-
|
@philippedegeer and @cheese202 , could you pls help to understand how were you able to get NPU installed inside the docker container as it is based on debian with package libtbb12 not available unlike in Ubuntu. For me intel-driver-compiler-npu and intel-level-zero-npu are unable to install inside the container. Though my host is able to detect NPU, the container is not able to do so. Did you add any PR for this or could you please share manual steps. |
Beta Was this translation helpful? Give feedback.
-
|
@architag21 , if my memory serves me correctly, I used the following commands, inside de container, to obtain de libtbb12. wget http://ftp.de.debian.org/debian/pool/main/o/onetbb/libtbb12_2021.8.0-2_amd64.deb I am positive that I was actually using the NPU, with the ssdlite model being the only functional one. This is where I left my testing. I did not add any PR. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @philippedegeer for your input. I tried using the same to obtain libtbb12 but found there are lot more dependencies just like the one you added above. For sure that is one way but I found that a new 0.16 release has been created which used Debian12 as the base image for this container. Using that image, libtbb12 can be installed directly through apt install command, and post that one can wget the Intel drivers and run dpkg -i *.deb. So either way NPU should work- Getting all dependencies through the links you provided in debian 11 OR apt install in debian 12 release. |
Beta Was this translation helpful? Give feedback.
-
|
I was able to install Intel NPU Drivers into my host (Proxmox) and pass it through utilizing the Proxmox GUI Device Pass Through (/dev/accel/accel0) and also added this into my container config (included - /dev/accel/accel0 arg under devices:). I have no way to confirm the % utilization utilizing the NPU argument in the frigate configuration - but inference speed gets around 18 ms. If I utilize GPU argument, it goes down to 8ms. If utilize CPU (I have a core ultra 125H) it goes to 4ms inference time. Proxmox GUI CPU Utilization percentage increases quite a lot and I am allowing 12 cores to the frigate container - increases from 3-4% to 10-15%. I am yet to plug a smart plug to check how many watts my Beeling SEi14 consumes utilizing NPU vs GPU - but looking at inference times alone makes me believe that I don't have to install Intel NPU Drivers inside the container to make it work - it is enough to pass through the hardware and configure it in the config.yml of the frigate container |
Beta Was this translation helpful? Give feedback.
-
|
Here's how I made the integrated NPU on my Intel Core Ultra 5 245K work with Frigate. Other distros should work as well, assuming DISCLAIMER: Tested inside of an LXC container on Proxmox VE 8.3.4, with an Intel Core Ultra 5 245K, kernel DISCLAIMER 2: This changes files inside of the Frigate container image. This means that any re-creates of the container or image updates of the container will erase these changes. You'll need to redo this process, starting from step 3. Hopefully Frigate will be updated in the near future to support Arrow Lake NPUs out of the box without needing this to be done. I would have tried to make my own image but I'm currently unable to figure out how to recreate the image builder process. These are the steps I needed to perform:
--- End of steps and additional notes:--- I realize these steps are similar to what has been posted before but only on 100 series Intel processors. It seems the currently distributed OpenVINO version works for that generation of product but not Arrow Lake/200 series. I do plan to test switching from the I had tried both only updating OpenVINO inside the container and also only installing the NPU packages inside the container, and each of them by themselves would not work. Updating only OpenVINO returned device errors, while installing only the NPU packages kept returning the constant "no available devices error". In my testing, the inference time on the Core Ultra 5 245K iGPU is ~4ms, while the NPU returns times in the range of 12-20ms. In my setup I'm not noticing a drastic difference in performance to be concerned, but it's worth noting. Power consumption for the whole system (measured from the wall with a Sonoff S31 running ESPHome) decreased by 2-3 watts, down from just shy of 60, to almost as low as 55 watts. I am willing to consider that power usage difference down to simple fluctuations through the day. In my setup I'm choosing to dedicate the NPU entirely for Frigate inference, so any other applications that run with OpenVINO use the iGPU (such as Immich with its smart search and facial recognition features). I haven't tried any of the semantic search or generative AI features in Frigate so I'm not sure how this will behave being ran on an NPU. Here is my proof that OpenVINO is making use of the NPU for inference. Note that the My camera setup: EDIT: Forgot to mention that the NPU |
Beta Was this translation helpful? Give feedback.
-
|
no, it would just be built in to the main image |
Beta Was this translation helpful? Give feedback.
-
@SamTheGreatCoder are you able to run VAINFO command on proxmox host? |
Beta Was this translation helpful? Give feedback.
-
@lutfijw When installed, I try to avoid installing any packages on the Proxmox host itself, and instead prefer to use LXC containers where possible. I keep a second container shutdown running a rolling release of either Alpine or Arch for troubleshooting anything graphics/NPU related. What error are you seeing? I'd like to try and help troubleshoot where possible. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @SamTheGreatCoder ,
|
Beta Was this translation helpful? Give feedback.
-
|
Bytheby, I was able to get a bit of a performance boost using the bookworm backports to upgrade the kernel and install the latest intel drivers. At the time of writing, current intel npu driver version is 1.23, from some testing, it seems 1.17 is the newest that can be installed with the shipped kernel in frigate 0.16 Context, I'm mostly just messing around to see what I can do, maybe this will completely break frigate for you, it didn't for me. My system is a best buy prebuilt I got for home assistant. Has an Intel Core 7 Ultra 265k, no dGPU. I'm running ultralytics pretrained yolo12n, with 1.23 drivers my inference times dropped to ~45ms from the ~75ms I had on 1.17 drivers. Still not amazing, but good enough for my 3 cameras, I think. **after running for a overnight, average NPU usage measured from host increased significantly and inference speed dropped to ~25ms... no idea what happened there, but the improvement persist through reboots. To verify stability, I used a fresh container, didn't change my linked volumes, just deleted the container image and rebuilt it. The following is what I did, not properly configured for a script, more like my notes, but basically copy/paste, need to add some text to sources, I used vim here. Some items aren't strictly necessary (file removals, etc.) but better reliable and reproducible than simple. |
Beta Was this translation helpful? Give feedback.
-
|
Here is my nice AI Generated Script to allow frigate NPU to work - it downloads the files locally then pushes them in. #!/bin/bash
# Intel NPU Driver Installation Script for Frigate Docker Container
# Updated for v1.17.0 (June 2025) and OpenVINO 2025.1 compatibility
# Downloads drivers on host, then copies into container
set -e # Exit on any error
CONTAINER_NAME="frigate"
DOWNLOAD_DIR="$HOME/npu-drivers"
echo "🚀 Installing Intel NPU drivers v1.17.0 in Frigate container..."
# Check if container exists and is running
if ! docker ps | grep -q "$CONTAINER_NAME"; then
echo "❌ Frigate container not found or not running"
echo "Make sure container '$CONTAINER_NAME' is running with: docker compose up -d"
exit 1
fi
echo "📥 Downloading NPU drivers to host system..."
# Create download directory
mkdir -p "$DOWNLOAD_DIR"
cd "$DOWNLOAD_DIR"
# Clean up any existing packages
rm -f *.deb
# Download Intel NPU drivers v1.17.0 (Ubuntu 22.04 - most compatible)
echo " • Intel NPU driver v1.17.0..."
wget -q https://github.com/intel/linux-npu-driver/releases/download/v1.17.0/intel-driver-compiler-npu_1.17.0.20250508-14912879441_ubuntu22.04_amd64.deb
wget -q https://github.com/intel/linux-npu-driver/releases/download/v1.17.0/intel-level-zero-npu_1.17.0.20250508-14912879441_ubuntu22.04_amd64.deb
# Download Level Zero v1.21.9 (latest compatible)
echo " • Level Zero v1.21.9..."
wget -q https://github.com/oneapi-src/level-zero/releases/download/v1.21.9/level-zero_1.21.9+u22.04_amd64.deb
# Download dependencies from Debian repos
echo " • Dependencies..."
wget -q http://ftp.us.debian.org/debian/pool/main/o/onetbb/libtbb12_2021.8.0-2_amd64.deb
wget -q http://ftp.us.debian.org/debian/pool/main/o/onetbb/libtbbbind-2-5_2021.8.0-2_amd64.deb
wget -q http://ftp.us.debian.org/debian/pool/main/o/onetbb/libtbbmalloc2_2021.8.0-2_amd64.deb
wget -q http://ftp.us.debian.org/debian/pool/main/h/hwloc/libhwloc15_2.9.0-1_amd64.deb
echo "📦 Copying packages into Frigate container..."
# Copy all .deb files to container
for deb_file in *.deb; do
echo " • Copying $deb_file..."
docker cp "$deb_file" "$CONTAINER_NAME:/tmp/"
done
echo "🔧 Installing packages in container..."
# Install packages inside container
docker exec "$CONTAINER_NAME" bash -c "
cd /tmp
echo 'Removing any conflicting packages...'
dpkg --purge --force-remove-reinstreq intel-driver-compiler-npu intel-level-zero-npu level-zero 2>/dev/null || true
echo 'Installing NPU packages...'
dpkg -i *.deb || {
echo 'Fixing dependencies...'
apt update -qq
apt --fix-broken install -y -qq
dpkg -i *.deb
}
echo 'Updating library cache...'
ldconfig
echo 'Cleaning up temp files...'
rm -f *.deb
"
echo "🔧 Updating OpenVINO in container..."
# Update OpenVINO with --break-system-packages since it's a container
docker exec "$CONTAINER_NAME" bash -c "
pip install --upgrade --quiet --break-system-packages openvino==2025.1.0
"
echo "🧪 Testing NPU detection..."
# Test NPU detection
docker exec "$CONTAINER_NAME" python3 -c "
try:
from openvino import Core
core = Core()
devices = core.available_devices
print('Available OpenVINO devices in Frigate:', devices)
if 'NPU' in devices:
print('✅ SUCCESS: NPU detected in Frigate container!')
try:
npu_name = core.get_property('NPU', 'FULL_DEVICE_NAME')
print(f'NPU info: {npu_name}')
except:
print('NPU detected but detailed info unavailable')
else:
print('❌ NPU not detected. Available devices:', devices)
print('Check that /dev/accel/accel0 is mounted in container')
except Exception as e:
print(f'❌ OpenVINO test failed: {e}')
"
# Check NPU device access in container
echo "🔍 Checking NPU device access..."
docker exec "$CONTAINER_NAME" bash -c "
if [ -e /dev/accel/accel0 ]; then
echo '✅ NPU device /dev/accel/accel0 is accessible'
ls -la /dev/accel/accel0
else
echo '❌ NPU device not found - check Docker device passthrough'
echo 'Add to docker-compose.yaml:'
echo ' devices:'
echo ' - /dev/accel:/dev/accel'
fi
"
echo "🔄 Restarting Frigate container to complete setup..."
docker restart "$CONTAINER_NAME"
echo "⏳ Waiting for container to restart..."
sleep 8
echo "🧪 Final NPU verification..."
docker exec "$CONTAINER_NAME" python3 -c "
from openvino import Core
core = Core()
devices = core.available_devices
print('Final test - Available devices:', devices)
if 'NPU' in devices:
print('🎉 NPU successfully installed and working!')
else:
print('❌ NPU not detected after restart')
"
echo ""
echo "✅ Installation complete!"
echo ""
echo "📝 Next steps:"
echo "1. Update your Frigate config.yaml detector section:"
echo " detectors:"
echo " openvino:"
echo " type: openvino"
echo " device: NPU # Changed from GPU to NPU"
echo ""
echo "2. Restart Frigate to use NPU:"
echo " docker restart frigate"
echo ""
echo "3. Check Frigate logs for NPU usage:"
echo " docker logs frigate | grep -i npu"
echo ""
echo "💾 Downloaded drivers saved in: $DOWNLOAD_DIR"
echo "⚠️ NOTE: These container changes are temporary and will be lost on container recreation."
echo " Re-run this script after any Frigate updates."
# Clean up download directory if desired
read -p "🗑️ Delete downloaded drivers? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
rm -rf "$DOWNLOAD_DIR"
echo "✅ Cleaned up downloaded files"
else
echo "📁 Drivers saved in $DOWNLOAD_DIR for future use"
fi |
Beta Was this translation helpful? Give feedback.
-
|
Also -> totally not worth it as far as I can tell - appears to be slow :) 
|
Beta Was this translation helpful? Give feedback.
-
|
yeah, that is why we have been slow to implement it |
Beta Was this translation helpful? Give feedback.
-
|
I kind of assumed it would be as good as a coral or something... APPARENTLY NOT! :) |
Beta Was this translation helpful? Give feedback.
-
|
Yolonas is much larger than the default coral model, it should run mobilenet v2 similarly |
Beta Was this translation helpful? Give feedback.
-
|
Just adding I've followed bits of both guides and got the NPU working (265K) and inference speed from from 24ms (google coral) to 4ms. 12 cameras. Not really typical results as the server is very over powered. |
Beta Was this translation helpful? Give feedback.
-
|
This means it is running on the GPU, not NPU. NPU has its own |
Beta Was this translation helpful? Give feedback.
-
|
Thanks all, I managed to get it fully working in Unprivileged LXC container after installing the intel GPU Driver in LXC. Now frigate is fully use the GPU, and the best thing is the Intel Compute Engine (the NPU, I think) is automatically activated whenever detection is happen even though I only enable GPU in Frigate.
|
Beta Was this translation helpful? Give feedback.
-
|
For everyone who are struggling to get Frigate running using GPU/NPU on latest Intel Arrow Lake Architecture, I follow the guide as outlined at https://dgpu-docs.intel.com/driver/client/overview.html and install the Intel GPU driver in LXC container. |
Beta Was this translation helpful? Give feedback.
-
|
Now I have some weird issues. The Frigate system metric always shown GPU Utilization as 0%, but the intel_gpu_top showing activities. I am not sure the detection is running on GPU or NPU or CPU.
|
Beta Was this translation helpful? Give feedback.
-
|
Based on that it's running on the GPU |
Beta Was this translation helpful? Give feedback.
-
|
I upgrade shell script from jeeftor Performance boost up. Inference time below 10ms. if use on npu. My system test on Intel Core Ultra7 258v.
#!/bin/bash
# Intel NPU Driver Installation Script for Frigate Docker Container
# Updated for v1.19.0 (Aug 2025), Level Zero v1.22.4, and OpenVINO 2025.2.0
# Checks for local .deb files first, only downloads if missing
# Allows OpenVINO version override with OPENVINO_VER env var
set -e # Exit on any error
# Auto-detect Frigate container name from image
CONTAINER_NAME=$(docker ps --filter ancestor=ghcr.io/blakeblackshear/frigate:0.16.0-rc2 --format '{{.Names}}' | head -n1)
# Get absolute path of this script
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PATCH_FILE="$SCRIPT_DIR/openvino.py"
# Download directory
DOWNLOAD_DIR="$HOME/npu-drivers"
mkdir -p "$DOWNLOAD_DIR"
cd "$DOWNLOAD_DIR"
# Default OpenVINO version (can be overridden via env var)
OPENVINO_VER="${OPENVINO_VER:-2025.2.0}"
echo "🚀 Installing Intel NPU drivers v1.19.0 (Ubuntu22.04) in Frigate container..."
echo "Using OpenVINO version: $OPENVINO_VER"
# Check if container exists and is running
if ! docker ps | grep -q "$CONTAINER_NAME"; then
echo "❌ Frigate container not found or not running"
echo "Make sure container '$CONTAINER_NAME' is running with: docker compose up -d"
exit 1
fi
# Intel NPU driver package list (Ubuntu 22.04 build)
PKGS=(
"intel-driver-compiler-npu_1.19.0.20250707-16111289554_ubuntu22.04_amd64.deb"
"intel-fw-npu_1.19.0.20250707-16111289554_ubuntu22.04_amd64.deb"
"intel-level-zero-npu_1.19.0.20250707-16111289554_ubuntu22.04_amd64.deb"
)
# Level Zero runtime (updated to v1.22.4)
PKGS+=("level-zero_1.22.4+u22.04_amd64.deb")
# Dependencies from Debian repos
PKGS+=(
"libtbb12_2021.8.0-2_amd64.deb"
"libtbbbind-2-5_2021.8.0-2_amd64.deb"
"libtbbmalloc2_2021.8.0-2_amd64.deb"
"libhwloc15_2.9.0-1_amd64.deb"
)
# URLs for each package
URLS=(
"https://github.com/intel/linux-npu-driver/releases/download/v1.19.0/${PKGS[0]}"
"https://github.com/intel/linux-npu-driver/releases/download/v1.19.0/${PKGS[1]}"
"https://github.com/intel/linux-npu-driver/releases/download/v1.19.0/${PKGS[2]}"
"https://github.com/oneapi-src/level-zero/releases/download/v1.22.4/${PKGS[3]}"
"http://ftp.us.debian.org/debian/pool/main/o/onetbb/${PKGS[4]}"
"http://ftp.us.debian.org/debian/pool/main/o/onetbb/${PKGS[5]}"
"http://ftp.us.debian.org/debian/pool/main/o/onetbb/${PKGS[6]}"
"http://ftp.us.debian.org/debian/pool/main/h/hwloc/${PKGS[7]}"
)
# Download missing files
for i in "${!PKGS[@]}"; do
if [[ -f "${PKGS[$i]}" ]]; then
echo "📥 Found local ${PKGS[$i]}, skipping download"
else
echo "📥 Downloading ${PKGS[$i]}..."
wget -q "${URLS[$i]}"
fi
done
echo "📦 Copying packages into Frigate container..."
for deb_file in "${PKGS[@]}"; do
docker cp "$deb_file" "$CONTAINER_NAME:/tmp/"
done
echo "• Installing packages in container..."
docker exec "$CONTAINER_NAME" bash -c "
cd /tmp
echo 'Removing any conflicting packages...'
dpkg --purge --force-remove-reinstreq intel-fw-npu intel-driver-compiler-npu intel-level-zero-npu level-zero 2>/dev/null || true
echo 'Installing NPU packages...'
dpkg -i *.deb || {
echo 'Fixing dependencies...'
apt update -qq
apt --fix-broken install -y -qq
dpkg -i *.deb
}
echo 'Updating library cache...'
ldconfig
echo 'Cleaning up temp files...'
rm -f *.deb
"
echo "🔧 Updating OpenVINO in container..."
docker exec "$CONTAINER_NAME" bash -c "
pip install --upgrade --quiet --break-system-packages --resume-retries 5 --timeout 120 openvino==${OPENVINO_VER}
"
# --- Patch Frigate's openvino.py ---
if [ -f "$PATCH_FILE" ]; then
echo "📦 Copying patched Frigate openvino.py into container..."
docker exec "$CONTAINER_NAME" bash -c "
if [ -f /opt/frigate/frigate/detectors/plugins/openvino.py ]; then
cp /opt/frigate/frigate/detectors/plugins/openvino.py \
/opt/frigate/frigate/detectors/plugins/openvino.py.bak
fi
"
docker cp "$PATCH_FILE" "$CONTAINER_NAME:/opt/frigate/frigate/detectors/plugins/openvino.py"
echo "🔧 Patched Frigate openvino.py applied"
else
echo "❌ No patched openvino.py found at $PATCH_FILE, skipping patch"
fi
echo "🧪 Testing NPU detection..."
docker exec "$CONTAINER_NAME" python3 -c "
try:
from openvino import Core
core = Core()
devices = core.available_devices
print('Available OpenVINO devices in Frigate:', devices)
if 'NPU' in devices:
print('✅ SUCCESS: NPU detected in Frigate container!')
try:
npu_name = core.get_property('NPU', 'FULL_DEVICE_NAME')
print(f'NPU info: {npu_name}')
except:
print('NPU detected but detailed info unavailable')
else:
print('❌ NPU not detected. Available devices:', devices)
except Exception as e:
print(f'❌ OpenVINO test failed: {e}')
"
echo "🔍 Checking NPU device access..."
docker exec "$CONTAINER_NAME" bash -c "
if [ -e /dev/accel/accel0 ]; then
echo '✅ NPU device /dev/accel/accel0 is accessible'
ls -la /dev/accel/accel0
else
echo '❌ NPU device not found - check Docker device passthrough'
fi
"
echo "🔄 Restarting Frigate container to complete setup..."
docker restart "$CONTAINER_NAME"
sleep 8
echo "🧪 Final NPU verification..."
docker exec "$CONTAINER_NAME" python3 -c "
from openvino import Core
core = Core()
devices = core.available_devices
print('Final test - Available devices:', devices)
if 'NPU' in devices:
print('✅ NPU successfully installed and working!')
else:
print('❌ NPU not detected after restart')
"
echo ""
echo "✅ Installation complete!"
echo ""
echo "📝 Next steps:"
echo "1. Update your Frigate config.yaml detector section:"
echo " detectors:"
echo " openvino:"
echo " type: openvino"
echo " device: NPU # Changed from GPU to NPU"
echo ""
echo "2. Restart Frigate to use NPU:"
echo " docker restart frigate"
echo ""
echo "3. Check Frigate logs for NPU usage:"
echo " docker logs frigate | grep -i npu"
echo ""
echo "💾 Downloaded drivers saved in: $DOWNLOAD_DIR"
echo "⚠️ NOTE: These container changes are temporary and will be lost on container recreation."
echo " Re-run this script after any Frigate updates."
# Clean up download directory if desired
read -p "🗑️ Delete downloaded drivers? (y/N): " -n 1 -r
echo
if [[ $REPLY =~ ^[Yy]$ ]]; then
rm -rf "$DOWNLOAD_DIR"
echo "✅ Cleaned up downloaded files"
else
echo "📁 Drivers saved in $DOWNLOAD_DIR for future use"
fi
''' |
Beta Was this translation helpful? Give feedback.
-
|
ubuntu 24.04, I created docker image. download this cluangar/frigate:0.16.0-rc2-npu |
Beta Was this translation helpful? Give feedback.
-
|
Will try your docker image. While I am still using GPU engine, intel_gpu_top is showing frigate detector engine is always using Compute Engine, I believe it is NPU. |
Beta Was this translation helpful? Give feedback.
-
|
I think that openvino fallback some operation to gpu or cpu. check latency It must reduce. |
Beta Was this translation helpful? Give feedback.
-
What kernel are you running? You need a recent kernel - you could be using ubuntu with a 6.8 kernel thats no dice but change to 6.14 and you are good... |
Beta Was this translation helpful? Give feedback.
-
|
Yes. I used ubuntu 24.04 with kernel 6.14 |
Beta Was this translation helpful? Give feedback.
Here's how I made the integrated NPU on my Intel Core Ultra 5 245K work with Frigate. Other distros should work as well, assuming
intel_vpuis supported by the kernel and you have a means of installing theintel-fw-npupackage.DISCLAIMER: Tested inside of an LXC container on Proxmox VE 8.3.4, with an Intel Core Ultra 5 245K, kernel
6.8.12-8-pve. Frigate 0.15.0 Release, Intel Linux NPU Driver v1.13.0. I also tested this on Ubuntu 24.04 (skipping over Docker nested inside an LXC container) on the same hardware and it worked the same. I only tested the default SSDlite Mobilenet V2 model. See below for my Frigateconfig.ymlfile.DISCLAIMER 2: This changes files inside of the Frigate contain…