High CPU usage with GPU encoding.

[sergiobaiao]

Hi there everyone. I'm setting up frigate to use my gpu for encoding/decoding ffmpeg (i have a tpu for inference). This is my working config.yml as of now:

config.yaml.txt

and my docker-compose.yml:
docker-compose.yml.txt

running nvidia-smi from the host:
servidorcoral:~$ nvidia-smi
Tue May 27 18:50:56 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.153.02 Driver Version: 570.153.02 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 3050 Off | 00000000:04:00.0 On | N/A |
| 37% 47C P0 25W / 70W | 3650MiB / 6144MiB | 6% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 2272774 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 232MiB |
| 0 N/A N/A 2272785 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 220MiB |
| 0 N/A N/A 2272796 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 216MiB |
| 0 N/A N/A 2272807 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 228MiB |
| 0 N/A N/A 2272818 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 228MiB |
| 0 N/A N/A 2272829 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 216MiB |
| 0 N/A N/A 2272851 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 224MiB |
| 0 N/A N/A 2272906 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 220MiB |
| 0 N/A N/A 2274155 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 97MiB |
| 0 N/A N/A 2274163 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274183 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274187 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274191 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274196 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274200 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274204 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 97MiB |
| 0 N/A N/A 2274212 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274232 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 99MiB |
| 0 N/A N/A 2274236 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274241 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 99MiB |
| 0 N/A N/A 2274245 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274250 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274254 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274258 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
| 0 N/A N/A 2274281 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 101MiB |
+-----------------------------------------------------------------------------------------+

and from the container:

servidorcoral:~$ docker exec -it frigate nvidia-smi
Tue May 27 18:51:29 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.153.02 Driver Version: 570.153.02 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 3050 Off | 00000000:04:00.0 On | N/A |
| 37% 48C P0 27W / 70W | 3927MiB / 6144MiB | 10% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+

Even with everything apparently correctly configured, this is how my htop looks like:

Does anyone have an idea on how to fix this?

[sergiobaiao]

as an added note, i've already tried the tensorrt image and had the same behaviour, even after reindexing.

[hawkeye217]

The processes using CPU are for the embeddings manager, which is responsible for Frigate's enrichments like Semantic Search, Face Recognition, and LPR. If you want to use hardware acceleration, you must use the appropriate Docker image for your hardware.

https://deploy-preview-16390--frigate-docs.netlify.app/configuration/hardware_acceleration_enrichments

[NickM-27]

Your config has a handful of issues with it.

1. You're transcoding your go2rtc stream to h264, and then again transcoding it to h264 in frigate? I don't see a reason to transcode it at all to be honest, surely your camera supports h264 output? But even then, transcoding twice is entirely unnecessary
2. Global args should not include hwaccel args.
3. You need to use tensorrt image for the GPU to be used for semantic search

[sergiobaiao]

1 -You're transcoding your go2rtc stream to h264, and then again transcoding it to h264 in frigate? I don't see a reason to transcode it at all to be honest, surely your camera supports h264 output? But even then, transcoding twice is entirely unnecessary

what should i change here? The cameras really support h264

2-Global args should not include hwaccel args.
removed that and nothing changed...

3-You need to use tensorrt image for the GPU to be used for semantic search
i'm running it as we speak, same behavior

have been trying everything, mix and matching configs to make this work

[NickM-27]

what should i change here? The cameras really support h264

1. change your camera to h264
2. change #video=h264#hardware to #video=copy
3. remove your manual recording output args and use a preset instead

removed that and nothing changed...

didn't expect it to change anything it was just incorrect

i'm running it as we speak, same behavior

what behavior are you referring to exactly? re-indexing will use a lot of CPU, it is processing all of the images as fast as possible. It will just use more CPU if running on the CPU. Once re-indexing is done and disabled, semantic search should use considerably less CPU than when running on CPU. re-indexing is not a good way to gauge if it is working correctly or not.

[sergiobaiao]

i'll change the #video=copy lines...

the behavior i'm referring to is that after re-indexing, i'm having GPU usage at 1117% while CPU usage is at 7080%
it's definitively falling back to CPU somehow.

[NickM-27]

you can use the nvtop tool to check if it is running on GPU or CPU

[sergiobaiao]

This is how my System Metrics page looks:

my htop at the same time:

and my nvtop:

See the difference between the CPU and GPU usage?

[sergiobaiao]

now i'm having a different and weird issue. Using the same URLs from go2rtc (without the #video=copy) i'm having problems with authentication. My password does have an @ symbol and it's url encoded (i'm using %40 instead of @). The same url worked with go2rtc, but directly it gives 403

[hawkeye217]

To be clear, the live --> streams directive must use go2rtc stream names, not urls. Note that the go2rtc streams will not utilize resources if they are not actively being consumed.

[NickM-27]

The passwords in the urls in Frigate should not be URL encoded, Frigate handles that.

[sergiobaiao]

ok, removed the live streams from config. still cannot use the same urls from go2rtc directly as camera streams. I've seen somewhere it may be an old ffmpeg bug with encoded urls.

[NickM-27]

There is no bug, the URLs should not be encoded in the config as Frigate applies URL encoding