Multiple ONNX detectors / performance

[nsprawling]

Is there a performance benefit to a higher / lower number of ONNX detectors?
I have a 3090 24GB & dual 20c Xeon 6148.

[inkeep-ai-assistant[bot]]

Defining multiple ONNX detectors in your Frigate configuration allows detection requests to be handled in parallel, which can improve throughput, especially when you have many camera streams or high frame rates(1). For Nvidia GPUs like your 3090, users have observed that using several ONNX detectors can lead to better scaling and performance, as each detector runs in a dedicated process and pulls from a common queue(2). This approach can be more efficient and use less memory compared to a single ONNX-TensorRT detector, while still leveraging the GPU effectively(2). However, the optimal number of detectors depends on your hardware and workload, so some experimentation may be required for best results(2).

📚 Sources:

1. Supported Hardware
2. [HW Accel Support]: onnx detector performance drop

---

Have another question? Just tag @inkeep.

[hawkeye217]

If you are seeing skipped detections in your Camera Metrics page, adding another detector is recommended. If you have relatively low activity, you'd benefit from a single detection process.